[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:02 dist-test-slave-dn4n krb5kdc[12154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:02 dist-test-slave-dn4n krb5kdc[12154](info): set up 2 sockets
Sep 25 02:27:02 dist-test-slave-dn4n krb5kdc[12154](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 02:27:03.210594 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:03.312681 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:37425
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:37425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:03.385810 12167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:03.385982 12167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:03.386001 12167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:03.387006 12167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:03.387041 12167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:03.387055 12167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:03.387096 12167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:03.387117 12167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:03.388465 12167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:37425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:37425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:03.388687 12167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:03.388881 12167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:03.393544 12175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:03.393563 12173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:03.393539 12167 server_base.cc:1034] running on GCE node
W20260925 02:27:03.393517 12172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:03.393975 12167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:03.394229 12167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:03.395372 12167 hybrid_clock.cc:648] HybridClock initialized: now 1790303223395360 us; error 24 us; skew 500 ppm
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:03.397671 12167 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:03.398054 12167 webserver.cc:460] Webserver started at http://127.11.220.254:34471/ using document root <none> and password file <none>
I20260925 02:27:03.398226 12167 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:03.398262 12167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:03.398340 12167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:03.399211 12167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "f65dbf452fae4bbaa4e263fe26fc2938"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "21744fdfeb3e84d8746dead755b59244"
server_key_iv: "a095e1cc9d9b0e8e84dcafd58bccf5db"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.399484 12167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "f65dbf452fae4bbaa4e263fe26fc2938"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "21744fdfeb3e84d8746dead755b59244"
server_key_iv: "a095e1cc9d9b0e8e84dcafd58bccf5db"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.400648 12167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:27:03.401258 12182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.401474 12167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.401556 12167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "f65dbf452fae4bbaa4e263fe26fc2938"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "21744fdfeb3e84d8746dead755b59244"
server_key_iv: "a095e1cc9d9b0e8e84dcafd58bccf5db"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.401631 12167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:03.410661 12167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:03.412248 12167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:03.412377 12167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:03.415979 12167 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:37425
I20260925 02:27:03.416023 12234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:37425 every 8 connection(s)
I20260925 02:27:03.416330 12167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:03.416647 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 12167
I20260925 02:27:03.416765 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:03.417250 12235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:03.417380 12147 external_mini_cluster.cc:1428] Setting key 0b5e65f5c114aef25e47c0fd7f9fb86e
I20260925 02:27:03.419991 12235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938: Bootstrap starting.
I20260925 02:27:03.420698 12235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:03.421033 12235 log.cc:824] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:03.421825 12235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938: No bootstrap required, opened a new log
I20260925 02:27:03.425235 12235 raft_consensus.cc:348] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65dbf452fae4bbaa4e263fe26fc2938" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37425 } }
I20260925 02:27:03.425393 12235 raft_consensus.cc:374] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:03.425428 12235 raft_consensus.cc:729] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f65dbf452fae4bbaa4e263fe26fc2938, State: Initialized, Role: FOLLOWER
I20260925 02:27:03.425547 12235 consensus_queue.cc:260] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65dbf452fae4bbaa4e263fe26fc2938" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37425 } }
I20260925 02:27:03.425619 12235 raft_consensus.cc:388] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:03.425650 12235 raft_consensus.cc:482] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:03.425697 12235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:03.426234 12235 raft_consensus.cc:504] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65dbf452fae4bbaa4e263fe26fc2938" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37425 } }
I20260925 02:27:03.426347 12235 leader_election.cc:302] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f65dbf452fae4bbaa4e263fe26fc2938; no voters:
I20260925 02:27:03.426496 12235 leader_election.cc:288] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:03.426544 12240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:03.426690 12240 raft_consensus.cc:686] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [term 1 LEADER]: Becoming Leader. State: Replica: f65dbf452fae4bbaa4e263fe26fc2938, State: Running, Role: LEADER
I20260925 02:27:03.426743 12235 sys_catalog.cc:564] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:03.426780 12240 consensus_queue.cc:237] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65dbf452fae4bbaa4e263fe26fc2938" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37425 } }
I20260925 02:27:03.427084 12241 sys_catalog.cc:455] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f65dbf452fae4bbaa4e263fe26fc2938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65dbf452fae4bbaa4e263fe26fc2938" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37425 } } }
I20260925 02:27:03.427066 12240 sys_catalog.cc:455] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f65dbf452fae4bbaa4e263fe26fc2938. Latest consensus state: current_term: 1 leader_uuid: "f65dbf452fae4bbaa4e263fe26fc2938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65dbf452fae4bbaa4e263fe26fc2938" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37425 } } }
I20260925 02:27:03.427287 12240 sys_catalog.cc:458] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:03.427453 12241 sys_catalog.cc:458] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:03.427758 12249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:03.428231 12249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:03.429675 12249 catalog_manager.cc:1347] Generated new cluster ID: 22824c972bbf453c91ea47985c0a0518
I20260925 02:27:03.429728 12249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:03.434303 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:03.418152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.11.220.254:37425)
0925 02:27:03.418300 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:03.418308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:03.418652 (+ 344us) server_negotiation.cc:407] Connection header received
0925 02:27:03.421392 (+ 2740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:03.421400 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:03.421464 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:03.421592 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:03.423516 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.424326 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.425856 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.426246 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.426339 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:03.430630 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:03.430633 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:03.430634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:03.430644 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:03.432409 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.432929 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.432931 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.432932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.433018 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.433310 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.433310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.433311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.433552 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:03.433684 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:03.434025 (+ 341us) server_negotiation.cc:299] Negotiation successful
0925 02:27:03.434131 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260925 02:27:03.444286 12249 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:03.444942 12249 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:03.448612 12249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f65dbf452fae4bbaa4e263fe26fc2938: Generated new TSK 0
I20260925 02:27:03.448755 12249 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:03.474654 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37425
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:03.549485 12261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:03.549691 12261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:03.549728 12261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:03.550755 12261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:03.550805 12261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:03.550843 12261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:03.552168 12261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37425
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:03.552405 12261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:03.552613 12261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:03.557030 12268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:03.557092 12267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:03.557176 12261 server_base.cc:1034] running on GCE node
W20260925 02:27:03.557017 12270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:03.557397 12261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:03.557576 12261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:03.558727 12261 hybrid_clock.cc:648] HybridClock initialized: now 1790303223558721 us; error 34 us; skew 500 ppm
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:03.560967 12261 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:03.561348 12261 webserver.cc:460] Webserver started at http://127.11.220.193:38657/ using document root <none> and password file <none>
I20260925 02:27:03.561556 12261 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:03.561622 12261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:03.561755 12261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:03.562660 12261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "62e97d1323ad4d41b4c524888dd119f4"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "1f85725719b64e3b9e101acfa4d80d66"
server_key_iv: "995ed5052cfae1ca37a687e638f25bb1"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.562978 12261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "62e97d1323ad4d41b4c524888dd119f4"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "1f85725719b64e3b9e101acfa4d80d66"
server_key_iv: "995ed5052cfae1ca37a687e638f25bb1"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.564127 12261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 02:27:03.564769 12277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.565021 12261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.565116 12261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "62e97d1323ad4d41b4c524888dd119f4"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "1f85725719b64e3b9e101acfa4d80d66"
server_key_iv: "995ed5052cfae1ca37a687e638f25bb1"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.565179 12261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:03.575313 12261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:03.576850 12261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:03.576983 12261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:03.577171 12261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:03.577419 12261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:03.577463 12261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.577492 12261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:03.577517 12261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.583073 12261 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:37163
I20260925 02:27:03.583132 12392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:37163 every 8 connection(s)
I20260925 02:27:03.583393 12261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:03.588600 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 12261
I20260925 02:27:03.588672 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:03.588815 12147 external_mini_cluster.cc:1428] Setting key 35af587d339c6411b43a30e58ef2274c
I20260925 02:27:03.591665 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:03.583983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:40319 (local address 127.11.220.254:37425)
0925 02:27:03.584020 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:03.584022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:03.584178 (+ 156us) server_negotiation.cc:407] Connection header received
0925 02:27:03.584625 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:03.584626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:03.584636 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:03.584682 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:03.585422 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.585859 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.586717 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.586888 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.586932 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:03.588787 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:03.588788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:03.588789 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:03.588796 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:03.590281 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.590760 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.590761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.590762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.590815 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.591069 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.591070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.591070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.591162 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:03.591207 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:03.591534 (+ 327us) server_negotiation.cc:299] Negotiation successful
0925 02:27:03.591596 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:27:03.591912 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:03.583975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:37425 (local address 127.11.220.193:40319)
0925 02:27:03.584090 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:03.584118 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:03.584543 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:03.584726 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:03.584729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:03.585201 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:03.585355 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:03.585359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.585926 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.585926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:03.586650 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:03.586652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.586949 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.586950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:03.587035 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:03.587452 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:03.587459 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:03.588689 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:03.590370 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:03.590377 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:03.590379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:03.590676 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:03.590876 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:03.590877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:03.590877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:03.591004 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:03.591215 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:03.591221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:03.591380 (+ 159us) client_negotiation.cc:769] Sending connection context
0925 02:27:03.591507 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 02:27:03.591663 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":67,"threads_started":1}
I20260925 02:27:03.592434 12393 heartbeater.cc:347] Connected to a master server at 127.11.220.254:37425
I20260925 02:27:03.592557 12393 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:03.592715 12393 heartbeater.cc:511] Master 127.11.220.254:37425 requested a full tablet report, sending...
I20260925 02:27:03.593158 12199 ts_manager.cc:194] Registered new tserver with Master: 62e97d1323ad4d41b4c524888dd119f4 (127.11.220.193:37163)
I20260925 02:27:03.593690 12199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:40319
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:03.611130 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37425
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:03.685163 12398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:03.685356 12398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:03.685388 12398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:03.686376 12398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:03.686429 12398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:03.686465 12398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:03.687737 12398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37425
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:03.687954 12398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:03.688155 12398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:03.692674 12406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:03.692675 12403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:03.692721 12398 server_base.cc:1034] running on GCE node
W20260925 02:27:03.692682 12404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:03.693010 12398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:03.693245 12398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:03.694396 12398 hybrid_clock.cc:648] HybridClock initialized: now 1790303223694374 us; error 25 us; skew 500 ppm
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:03.696645 12398 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:03.697026 12398 webserver.cc:460] Webserver started at http://127.11.220.194:34209/ using document root <none> and password file <none>
I20260925 02:27:03.697247 12398 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:03.697312 12398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:03.697434 12398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:03.698397 12398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "1a33e7e97c3e4009b6c4d464e1aa4301"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "f51e117d7484577dbfc619472f27a087"
server_key_iv: "09b143da0111191a8e81c539713fe8db"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.698729 12398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "1a33e7e97c3e4009b6c4d464e1aa4301"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "f51e117d7484577dbfc619472f27a087"
server_key_iv: "09b143da0111191a8e81c539713fe8db"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.699926 12398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:27:03.700620 12413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.700781 12398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:03.700870 12398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "1a33e7e97c3e4009b6c4d464e1aa4301"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "f51e117d7484577dbfc619472f27a087"
server_key_iv: "09b143da0111191a8e81c539713fe8db"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.700945 12398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:03.715229 12398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:03.717300 12398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:03.717437 12398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:03.717628 12398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:03.717891 12398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:03.717935 12398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.717967 12398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:03.717990 12398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.723512 12398 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:34101
I20260925 02:27:03.723564 12527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:34101 every 8 connection(s)
I20260925 02:27:03.723841 12398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:03.724818 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 12398
I20260925 02:27:03.724918 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:03.725087 12147 external_mini_cluster.cc:1428] Setting key df343b575eae7d5795ec336d050d8aad
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:03.734195 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:03.724499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:52885 (local address 127.11.220.254:37425)
0925 02:27:03.724561 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:03.724563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:03.724821 (+ 258us) server_negotiation.cc:407] Connection header received
0925 02:27:03.725426 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:03.725427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:03.725439 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:03.725490 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:03.726830 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.727316 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.728689 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.729021 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.729083 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:03.731626 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:03.731627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:03.731628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:03.731635 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:03.733034 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.733383 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.733385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.733385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.733433 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.733655 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.733656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.733656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.733756 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:03.733815 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:03.734046 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 02:27:03.734119 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 02:27:03.734341 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:03.724507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:37425 (local address 127.11.220.194:52885)
0925 02:27:03.724682 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:03.724726 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:03.725314 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:03.725593 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:03.725615 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:03.726448 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:03.726726 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:03.726733 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.727464 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.727465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:03.728613 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:03.728616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.729122 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.729124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:03.729319 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:03.729762 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:03.729771 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:03.731528 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:03.733118 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:03.733124 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:03.733126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:03.733317 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:03.733504 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:03.733504 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:03.733505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:03.733597 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:03.733818 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:03.733825 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:03.733936 (+ 111us) client_negotiation.cc:769] Sending connection context
0925 02:27:03.734018 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 02:27:03.734151 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
I20260925 02:27:03.734740 12528 heartbeater.cc:347] Connected to a master server at 127.11.220.254:37425
I20260925 02:27:03.734803 12528 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:03.734915 12528 heartbeater.cc:511] Master 127.11.220.254:37425 requested a full tablet report, sending...
I20260925 02:27:03.735212 12199 ts_manager.cc:194] Registered new tserver with Master: 1a33e7e97c3e4009b6c4d464e1aa4301 (127.11.220.194:34101)
I20260925 02:27:03.735561 12199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:52885
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:03.749045 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37425
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:03.824194 12533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:03.824358 12533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:03.824381 12533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:03.825507 12533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:03.825546 12533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:03.825590 12533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:03.827088 12533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37425
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:03.827294 12533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:03.827489 12533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:03.832129 12539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:03.832135 12538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:03.832145 12541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:03.832201 12533 server_base.cc:1034] running on GCE node
I20260925 02:27:03.832542 12533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:03.832798 12533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:03.833966 12533 hybrid_clock.cc:648] HybridClock initialized: now 1790303223833947 us; error 32 us; skew 500 ppm
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:03.836246 12533 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:03.836694 12533 webserver.cc:460] Webserver started at http://127.11.220.195:38841/ using document root <none> and password file <none>
I20260925 02:27:03.836934 12533 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:03.837002 12533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:03.837162 12533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:03.838182 12533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "0670d45c61b147f48021671843e24853"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "8a01a70557cd05c94f936f5498862a70"
server_key_iv: "f5af252df063d4050c9eafe077b3bc64"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.838536 12533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "0670d45c61b147f48021671843e24853"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "8a01a70557cd05c94f936f5498862a70"
server_key_iv: "f5af252df063d4050c9eafe077b3bc64"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.839763 12533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:03.840410 12548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.840564 12533 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:03.840652 12533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "0670d45c61b147f48021671843e24853"
format_stamp: "Formatted at 2026-09-25 02:27:03 on dist-test-slave-dn4n"
server_key: "8a01a70557cd05c94f936f5498862a70"
server_key_iv: "f5af252df063d4050c9eafe077b3bc64"
server_key_version: "encryptionkey@0"
I20260925 02:27:03.840730 12533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:03.848845 12533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:03.850370 12533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:03.850519 12533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:03.850711 12533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:03.850991 12533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:03.851037 12533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.851068 12533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:03.851091 12533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:03.856860 12533 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:36107
I20260925 02:27:03.856925 12662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:36107 every 8 connection(s)
I20260925 02:27:03.857201 12533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:03.862716 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 12533
I20260925 02:27:03.862811 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:03.862998 12147 external_mini_cluster.cc:1428] Setting key a02b8d2f7de72fe365b9457eb2ac005a
I20260925 02:27:03.865722 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:03.857867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:58547 (local address 127.11.220.254:37425)
0925 02:27:03.857918 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:03.857920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:03.858094 (+ 174us) server_negotiation.cc:407] Connection header received
0925 02:27:03.858485 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:03.858487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:03.858496 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:03.858539 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:03.859390 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.859837 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.860843 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.861026 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.861072 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:03.862980 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:03.862982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:03.862982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:03.862988 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:03.864439 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.864836 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.864837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.864837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.864891 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.865129 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.865130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.865130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.865235 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:03.865296 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:03.865550 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 02:27:03.865626 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 02:27:03.865859 12665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:03.857835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:37425 (local address 127.11.220.195:58547)
0925 02:27:03.857980 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:03.858010 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:03.858409 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:03.858592 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:03.858595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:03.859098 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:03.859325 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:03.859330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.859905 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.859905 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:03.860769 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:03.860771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.861087 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.861088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:03.861178 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:03.861531 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:03.861539 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:03.862894 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:03.864535 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:03.864542 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:03.864544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:03.864752 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:03.864966 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:03.864967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:03.864967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:03.865062 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:03.865300 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:03.865307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:03.865426 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 02:27:03.865515 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 02:27:03.865654 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260925 02:27:03.866262 12663 heartbeater.cc:347] Connected to a master server at 127.11.220.254:37425
I20260925 02:27:03.866348 12663 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:03.866509 12663 heartbeater.cc:511] Master 127.11.220.254:37425 requested a full tablet report, sending...
I20260925 02:27:03.866870 12199 ts_manager.cc:194] Registered new tserver with Master: 0670d45c61b147f48021671843e24853 (127.11.220.195:36107)
I20260925 02:27:03.867250 12199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:58547
I20260925 02:27:03.874737 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:27:03.894685 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:03.887926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49894 (local address 127.11.220.193:37163)
0925 02:27:03.888078 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:03.888087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:03.888096 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:03.888118 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:03.888119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:03.888132 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:03.888192 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:03.888757 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.889434 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.890147 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.890426 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.890478 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:03.892429 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:03.892437 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:03.892437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:03.892443 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:03.893737 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.894065 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.894067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.894067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.894115 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.894272 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.894273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.894273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.894380 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:03.894467 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:03.894522 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 02:27:03.894577 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
W20260925 02:27:03.895035 12306 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49894
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:27:03.914108 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:03.908672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49910 (local address 127.11.220.193:37163)
0925 02:27:03.908716 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:03.908717 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:03.908788 (+ 71us) server_negotiation.cc:407] Connection header received
0925 02:27:03.908838 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:03.908838 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:03.908847 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:03.908890 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:03.909285 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.909672 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.910242 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:03.910406 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:03.910460 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:03.912293 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:03.912294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:03.912295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:03.912300 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:03.913418 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.913683 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.913684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.913684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.913727 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:03.913827 (+ 100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:03.913827 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:03.913828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:03.913891 (+ 63us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:03.913958 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:03.914004 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 02:27:03.914053 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 02:27:03.914520 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 12261
I20260925 02:27:03.920595 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 12398
I20260925 02:27:03.924726 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 12533
I20260925 02:27:03.928874 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 12167
2026-09-25T02:27:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1693 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12677](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12677](info): set up 2 sockets
Sep 25 02:27:03 dist-test-slave-dn4n krb5kdc[12677](info): commencing operation
krb5kdc: starting...
W20260925 02:27:04.905838 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:04 dist-test-slave-dn4n krb5kdc[12677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:05.006814 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43655
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:43655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:05.084062 12690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:05.084260 12690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:05.084290 12690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:05.085348 12690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:05.085405 12690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:05.085426 12690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:05.085445 12690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:05.085470 12690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:05.086789 12690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:43655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:05.087023 12690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:05.087222 12690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:05.091795 12698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:05.091771 12696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:05.091784 12695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:05.092001 12690 server_base.cc:1034] running on GCE node
I20260925 02:27:05.092227 12690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:05.092545 12690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:05.093675 12690 hybrid_clock.cc:648] HybridClock initialized: now 1790303225093666 us; error 26 us; skew 500 ppm
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:05.100420 12690 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:05.100916 12690 webserver.cc:460] Webserver started at http://127.11.220.254:44127/ using document root <none> and password file <none>
I20260925 02:27:05.101135 12690 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:05.101203 12690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:05.101339 12690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:05.102315 12690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "2beb5de5fa58906665617f00149e18b0"
server_key_iv: "d4facaa544bf89aaea54ff52b8451df8"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.102658 12690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "2beb5de5fa58906665617f00149e18b0"
server_key_iv: "d4facaa544bf89aaea54ff52b8451df8"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.103873 12690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 02:27:05.104540 12705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.104712 12690 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.104802 12690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "2beb5de5fa58906665617f00149e18b0"
server_key_iv: "d4facaa544bf89aaea54ff52b8451df8"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.104882 12690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:05.125380 12690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:05.127389 12690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:05.127545 12690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:05.131453 12690 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:43655
I20260925 02:27:05.131479 12758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:43655 every 8 connection(s)
I20260925 02:27:05.131842 12690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:05.132714 12759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:05.134713 12759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab: Bootstrap starting.
I20260925 02:27:05.135281 12759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:05.135561 12759 log.cc:824] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:05.136191 12759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab: No bootstrap required, opened a new log
I20260925 02:27:05.137708 12759 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43655 } }
I20260925 02:27:05.137842 12759 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:05.137861 12759 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0ebbf205eaf49f3a012e0ac8b4ce3ab, State: Initialized, Role: FOLLOWER
I20260925 02:27:05.137953 12759 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43655 } }
I20260925 02:27:05.137990 12759 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:05.138046 12759 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:05.138078 12759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:05.138505 12759 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43655 } }
I20260925 02:27:05.138602 12759 leader_election.cc:302] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0ebbf205eaf49f3a012e0ac8b4ce3ab; no voters:
I20260925 02:27:05.138754 12759 leader_election.cc:288] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:05.138794 12762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:05.138974 12759 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:05.138991 12762 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [term 1 LEADER]: Becoming Leader. State: Replica: e0ebbf205eaf49f3a012e0ac8b4ce3ab, State: Running, Role: LEADER
I20260925 02:27:05.139063 12762 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43655 } }
I20260925 02:27:05.139541 12764 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0ebbf205eaf49f3a012e0ac8b4ce3ab. Latest consensus state: current_term: 1 leader_uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43655 } } }
I20260925 02:27:05.139624 12764 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:05.139945 12771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:05.139932 12763 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0ebbf205eaf49f3a012e0ac8b4ce3ab" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43655 } } }
I20260925 02:27:05.140018 12763 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:05.140445 12771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:05.140570 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 12690
I20260925 02:27:05.140641 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:05.140787 12147 external_mini_cluster.cc:1428] Setting key 01c177cfd072ba4c4f4b552a3eb4329a
I20260925 02:27:05.142226 12771 catalog_manager.cc:1347] Generated new cluster ID: cc2f5656eaeb47a499c44ea8872b95c6
I20260925 02:27:05.142275 12771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:05.151214 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.141369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55882 (local address 127.11.220.254:43655)
0925 02:27:05.141524 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.141532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.141565 (+ 33us) server_negotiation.cc:407] Connection header received
0925 02:27:05.141680 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.141685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.141758 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.141896 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:05.142687 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.143967 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.144900 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.145485 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.145597 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.147993 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:05.147996 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:05.147997 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:05.148007 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:05.149782 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.150132 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.150134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.150135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.150225 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.150416 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.150417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.150417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.150669 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:05.150800 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.150908 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.151018 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260925 02:27:05.155113 12771 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:05.155517 12771 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:05.160156 12771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0ebbf205eaf49f3a012e0ac8b4ce3ab: Generated new TSK 0
I20260925 02:27:05.160316 12771 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:05.185110 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43655
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:05.258072 12785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:05.258252 12785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:05.258273 12785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:05.259312 12785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:05.259348 12785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:05.259372 12785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:05.260715 12785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43655
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:05.260926 12785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:05.261138 12785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:05.265785 12791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:05.265820 12793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:05.265791 12790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:05.265952 12785 server_base.cc:1034] running on GCE node
I20260925 02:27:05.266166 12785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:05.266405 12785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:05.267555 12785 hybrid_clock.cc:648] HybridClock initialized: now 1790303225267543 us; error 27 us; skew 500 ppm
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:05.269889 12785 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:05.270280 12785 webserver.cc:460] Webserver started at http://127.11.220.193:35269/ using document root <none> and password file <none>
I20260925 02:27:05.270489 12785 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:05.270553 12785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:05.270700 12785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:05.271632 12785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "c82aab97f26e226bcc8335234002743e"
server_key_iv: "b655d3872b462ab93d6636719166e84d"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.271975 12785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "c82aab97f26e226bcc8335234002743e"
server_key_iv: "b655d3872b462ab93d6636719166e84d"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.273309 12785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:05.274037 12800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.274211 12785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:05.274304 12785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "c82aab97f26e226bcc8335234002743e"
server_key_iv: "b655d3872b462ab93d6636719166e84d"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.274376 12785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:05.289101 12785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:05.290632 12785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:05.290774 12785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:05.290970 12785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:05.291270 12785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:05.291317 12785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.291354 12785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:05.291378 12785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.296862 12785 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:33863
I20260925 02:27:05.296901 12914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:33863 every 8 connection(s)
I20260925 02:27:05.297200 12785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:05.298929 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 12785
I20260925 02:27:05.299023 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:05.299187 12147 external_mini_cluster.cc:1428] Setting key e20081bdd8440841e6a91f096a285e14
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
I20260925 02:27:05.309124 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.297880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:49859 (local address 127.11.220.254:43655)
0925 02:27:05.297934 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.297950 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.298118 (+ 168us) server_negotiation.cc:407] Connection header received
0925 02:27:05.299203 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.299204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.299217 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.299375 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:05.300713 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.301509 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.302982 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.303297 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.303357 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.306055 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:05.306057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:05.306058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:05.306065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:05.307778 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.308218 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.308219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.308219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.308266 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.308557 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.308557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.308558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.308645 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:05.308696 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.309000 (+ 304us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.309065 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 02:27:05.309351 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.297849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43655 (local address 127.11.220.193:49859)
0925 02:27:05.298003 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:05.298040 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:05.299046 (+ 1006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:05.299476 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:05.299480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:05.300358 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:05.300624 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.300629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.301597 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.301598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.302903 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.302906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.303418 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.303419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.303547 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.304117 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:05.304127 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:05.305958 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:05.307847 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:05.307854 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:05.307856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:05.308142 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:05.308330 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:05.308331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:05.308331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:05.308506 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:05.308704 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:05.308712 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:05.308867 (+ 155us) client_negotiation.cc:769] Sending connection context
0925 02:27:05.308967 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 02:27:05.309112 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260925 02:27:05.309922 12915 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43655
I20260925 02:27:05.310011 12915 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:05.310171 12915 heartbeater.cc:511] Master 127.11.220.254:43655 requested a full tablet report, sending...
I20260925 02:27:05.310637 12722 ts_manager.cc:194] Registered new tserver with Master: e5d59bf0573f4f6ca1e7eef9aedd4ed8 (127.11.220.193:33863)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
I20260925 02:27:05.311390 12722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:49859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:05.322880 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43655
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:05.396037 12920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:05.396206 12920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:05.396229 12920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:05.397300 12920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:05.397337 12920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:05.397361 12920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:05.398723 12920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43655
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:05.398880 12920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:05.399091 12920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:05.403782 12928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:05.403824 12926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:05.403790 12925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:05.403963 12920 server_base.cc:1034] running on GCE node
I20260925 02:27:05.404244 12920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:05.404505 12920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:05.405644 12920 hybrid_clock.cc:648] HybridClock initialized: now 1790303225405625 us; error 30 us; skew 500 ppm
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:05.408030 12920 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:05.408527 12920 webserver.cc:460] Webserver started at http://127.11.220.194:38029/ using document root <none> and password file <none>
I20260925 02:27:05.408780 12920 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:05.408841 12920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:05.408962 12920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:05.409907 12920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "01ce88b93d964545a65f0e5c1b7f08fc"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "6fd977e76db6214d3646952687f9980a"
server_key_iv: "8ffd933644b63a0b2f9ab453d09bf5d4"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.410262 12920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "01ce88b93d964545a65f0e5c1b7f08fc"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "6fd977e76db6214d3646952687f9980a"
server_key_iv: "8ffd933644b63a0b2f9ab453d09bf5d4"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.411474 12920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:27:05.412096 12935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.412271 12920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:05.412356 12920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "01ce88b93d964545a65f0e5c1b7f08fc"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "6fd977e76db6214d3646952687f9980a"
server_key_iv: "8ffd933644b63a0b2f9ab453d09bf5d4"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.412436 12920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:05.432919 12920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:05.434419 12920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:05.434552 12920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:05.434773 12920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:05.435039 12920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:05.435089 12920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.435119 12920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:05.435143 12920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.440402 12920 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:44901
I20260925 02:27:05.440471 13049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:44901 every 8 connection(s)
I20260925 02:27:05.440734 12920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:05.447218 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 12920
I20260925 02:27:05.447316 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:05.447474 12147 external_mini_cluster.cc:1428] Setting key 45f35dcd479c0b671c6cbf0cadd3b220
I20260925 02:27:05.449018 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.441377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:59221 (local address 127.11.220.254:43655)
0925 02:27:05.441438 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.441440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.441591 (+ 151us) server_negotiation.cc:407] Connection header received
0925 02:27:05.442425 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.442426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.442436 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.442482 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:05.442985 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.443422 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.444259 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.444421 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.444532 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.446274 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:05.446275 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:05.446275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:05.446281 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:05.447666 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.448120 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.448121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.448122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.448179 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.448419 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.448420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.448420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.448554 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:05.448624 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.448863 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.448935 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 02:27:05.449196 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.441341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43655 (local address 127.11.220.194:59221)
0925 02:27:05.441485 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:05.441520 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:05.442345 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:05.442536 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:05.442539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:05.442769 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:05.442925 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.442928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.443492 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.443492 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.444196 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.444197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.444545 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.444546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.444644 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.444998 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:05.445005 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:05.446172 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:05.447759 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:05.447766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:05.447769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:05.448025 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:05.448240 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:05.448241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:05.448241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:05.448345 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:05.448616 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:05.448624 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:05.448746 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 02:27:05.448842 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 02:27:05.448978 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260925 02:27:05.449752 13050 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43655
I20260925 02:27:05.449836 13050 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:05.450028 13050 heartbeater.cc:511] Master 127.11.220.254:43655 requested a full tablet report, sending...
I20260925 02:27:05.450371 12722 ts_manager.cc:194] Registered new tserver with Master: 01ce88b93d964545a65f0e5c1b7f08fc (127.11.220.194:44901)
I20260925 02:27:05.450728 12722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:59221
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:05.468569 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43655
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:05.541718 13055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:05.541886 13055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:05.541906 13055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:05.542899 13055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:05.542927 13055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:05.542950 13055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:05.544212 13055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43655
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:05.544361 13055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:05.544523 13055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:05.548684 13061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:05.548663 13063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:05.548676 13060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:05.548962 13055 server_base.cc:1034] running on GCE node
I20260925 02:27:05.549165 13055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:05.549376 13055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:05.550527 13055 hybrid_clock.cc:648] HybridClock initialized: now 1790303225550510 us; error 28 us; skew 500 ppm
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:05.552685 13055 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:05.553056 13055 webserver.cc:460] Webserver started at http://127.11.220.195:39515/ using document root <none> and password file <none>
I20260925 02:27:05.553262 13055 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:05.553321 13055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:05.553439 13055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:05.554340 13055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "4645e2b7358a4d9096d805c4c325ed0c"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "4946266930a549067b923f760a683e9c"
server_key_iv: "aa0be04e67a9696181d995824065df0a"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.554678 13055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "4645e2b7358a4d9096d805c4c325ed0c"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "4946266930a549067b923f760a683e9c"
server_key_iv: "aa0be04e67a9696181d995824065df0a"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.555763 13055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:05.556368 13070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.556541 13055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:05.556635 13055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "4645e2b7358a4d9096d805c4c325ed0c"
format_stamp: "Formatted at 2026-09-25 02:27:05 on dist-test-slave-dn4n"
server_key: "4946266930a549067b923f760a683e9c"
server_key_iv: "aa0be04e67a9696181d995824065df0a"
server_key_version: "encryptionkey@0"
I20260925 02:27:05.556712 13055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:05.571849 13055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:05.573496 13055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:05.573638 13055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:05.573830 13055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:05.574085 13055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:05.574132 13055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.574167 13055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:05.574190 13055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:05.579442 13055 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:46603
I20260925 02:27:05.579689 13185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:46603 every 8 connection(s)
I20260925 02:27:05.579982 13055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:05.582104 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 13055
I20260925 02:27:05.582180 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:05.582331 12147 external_mini_cluster.cc:1428] Setting key 636c0c431a8f632c51b8155c204214b6
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:05.588766 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.580762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:48119 (local address 127.11.220.254:43655)
0925 02:27:05.580837 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.580839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.581027 (+ 188us) server_negotiation.cc:407] Connection header received
0925 02:27:05.581846 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.581847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.581855 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.581911 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:05.582586 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.583263 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.584240 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.584429 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.584555 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.586408 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:05.586409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:05.586409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:05.586415 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:05.587719 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.588046 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.588047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.588047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.588093 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.588272 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.588272 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.588272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.588354 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:05.588396 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.588658 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.588711 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 02:27:05.588923 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.580742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43655 (local address 127.11.220.195:48119)
0925 02:27:05.580917 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:05.580953 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:05.581761 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:05.581961 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:05.581964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:05.582247 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:05.582487 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.582493 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.583340 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.583341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.584168 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.584171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.584552 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.584553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.584644 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.585020 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:05.585027 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:05.586322 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:05.587797 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:05.587802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:05.587804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:05.587989 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:05.588143 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:05.588143 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:05.588144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:05.588225 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:05.588399 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:05.588403 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:05.588558 (+ 155us) client_negotiation.cc:769] Sending connection context
0925 02:27:05.588631 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:27:05.588728 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260925 02:27:05.589376 13186 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43655
I20260925 02:27:05.589455 13186 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:05.589571 13186 heartbeater.cc:511] Master 127.11.220.254:43655 requested a full tablet report, sending...
I20260925 02:27:05.589896 12722 ts_manager.cc:194] Registered new tserver with Master: 4645e2b7358a4d9096d805c4c325ed0c (127.11.220.195:46603)
I20260925 02:27:05.590224 12722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:48119
I20260925 02:27:05.593669 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:27:05.599889 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.595163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55884 (local address 127.11.220.254:43655)
0925 02:27:05.595211 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.595213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.595312 (+ 99us) server_negotiation.cc:407] Connection header received
0925 02:27:05.595346 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.595347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.595354 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.595396 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:05.595872 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.596302 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.596973 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.597134 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.597181 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.598028 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:05.598029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:05.598030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:05.598034 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:05.599142 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.599397 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.599397 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.599398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.599437 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.599561 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.599562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.599562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.599663 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:05.599720 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.599797 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.599842 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 02:27:05.601747 12722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:05.603211 12722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:05.608567 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.605193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56158 (local address 127.11.220.195:46603)
0925 02:27:05.605343 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.605347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.605510 (+ 163us) server_negotiation.cc:407] Connection header received
0925 02:27:05.605546 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.605548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.605562 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.605625 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:05.605745 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.606599 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.607620 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.608206 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.608270 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.608360 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.608462 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.608502 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260925 02:27:05.608573 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.605036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:46603 (local address 127.0.0.1:56158)
0925 02:27:05.605437 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:05.605444 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:05.605483 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:05.605632 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:05.605633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:05.605634 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:05.605674 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.605675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.606692 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.606693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.607534 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.607537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.608280 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.608281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.608353 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.608364 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 02:27:05.608428 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:27:05.608497 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":2,"thread_start_us":56,"threads_started":1}
I20260925 02:27:05.609072 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.605036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:33863 (local address 127.0.0.1:58888)
0925 02:27:05.605256 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:05.605295 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:05.605354 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:05.605543 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:05.605545 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:05.605547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:05.605597 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.605612 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.606820 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.606821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.607809 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.607810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.608873 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.608874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.608930 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.608933 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:05.608979 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 02:27:05.609014 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":3,"thread_start_us":50,"threads_started":1}
I20260925 02:27:05.609174 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.605042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:44901 (local address 127.0.0.1:51160)
0925 02:27:05.605657 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:05.605667 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:05.605716 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:05.605860 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:05.605860 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:05.605862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:05.605978 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.605981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.606889 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.606889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.607808 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.607810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.608929 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.608930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.608983 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.608987 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:05.609039 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 02:27:05.609078 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":4,"thread_start_us":76,"threads_started":1}
I20260925 02:27:05.609205 13119 tablet_service.cc:1467] Processing CreateTablet for tablet 897c4b7d11724b3b9314a5c8bb5e47a6 (DEFAULT_TABLE table=test-table [id=29e2b983ebb3458c82d470cfe92438c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:05.609342 13199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.605243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51160 (local address 127.11.220.194:44901)
0925 02:27:05.605415 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.605419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.605717 (+ 298us) server_negotiation.cc:407] Connection header received
0925 02:27:05.605777 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.605779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.605792 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.605854 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:05.606075 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.606814 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.607899 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.608591 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.608930 (+ 339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.608988 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.609059 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.609098 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260925 02:27:05.609395 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.605201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58888 (local address 127.11.220.193:33863)
0925 02:27:05.605369 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.605372 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.605381 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:05.605428 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.605430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.605440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.605531 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:05.605692 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.606739 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.607990 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.608663 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.608734 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.608788 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.609287 (+ 499us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.609326 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":68,"threads_started":1}
I20260925 02:27:05.609525 13119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 897c4b7d11724b3b9314a5c8bb5e47a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:05.609838 12983 tablet_service.cc:1467] Processing CreateTablet for tablet 897c4b7d11724b3b9314a5c8bb5e47a6 (DEFAULT_TABLE table=test-table [id=29e2b983ebb3458c82d470cfe92438c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:05.609922 12848 tablet_service.cc:1467] Processing CreateTablet for tablet 897c4b7d11724b3b9314a5c8bb5e47a6 (DEFAULT_TABLE table=test-table [id=29e2b983ebb3458c82d470cfe92438c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:05.610117 12983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 897c4b7d11724b3b9314a5c8bb5e47a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:05.610211 12848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 897c4b7d11724b3b9314a5c8bb5e47a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:05.612314 13200 tablet_bootstrap.cc:492] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc: Bootstrap starting.
I20260925 02:27:05.612722 13201 tablet_bootstrap.cc:492] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c: Bootstrap starting.
I20260925 02:27:05.613014 13200 tablet_bootstrap.cc:654] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:05.613332 13201 tablet_bootstrap.cc:654] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:05.613415 13202 tablet_bootstrap.cc:492] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8: Bootstrap starting.
I20260925 02:27:05.613509 13200 log.cc:824] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:05.613834 13201 log.cc:824] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:05.614048 13202 tablet_bootstrap.cc:654] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:05.614614 13202 log.cc:824] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:05.614645 13201 tablet_bootstrap.cc:492] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c: No bootstrap required, opened a new log
I20260925 02:27:05.614698 13201 ts_tablet_manager.cc:1397] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:05.615253 13202 tablet_bootstrap.cc:492] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8: No bootstrap required, opened a new log
I20260925 02:27:05.615319 13200 tablet_bootstrap.cc:492] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc: No bootstrap required, opened a new log
I20260925 02:27:05.615329 13202 ts_tablet_manager.cc:1397] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:05.615381 13200 ts_tablet_manager.cc:1397] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 02:27:05.616746 13201 raft_consensus.cc:348] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4645e2b7358a4d9096d805c4c325ed0c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46603 } } peers { permanent_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33863 } } peers { permanent_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 44901 } }
I20260925 02:27:05.616926 13201 raft_consensus.cc:374] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:05.616961 13201 raft_consensus.cc:729] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4645e2b7358a4d9096d805c4c325ed0c, State: Initialized, Role: FOLLOWER
I20260925 02:27:05.617112 13201 consensus_queue.cc:260] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4645e2b7358a4d9096d805c4c325ed0c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46603 } } peers { permanent_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33863 } } peers { permanent_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 44901 } }
I20260925 02:27:05.617039 13202 raft_consensus.cc:348] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4645e2b7358a4d9096d805c4c325ed0c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46603 } } peers { permanent_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33863 } } peers { permanent_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 44901 } }
I20260925 02:27:05.617214 13202 raft_consensus.cc:374] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:05.617255 13202 raft_consensus.cc:729] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5d59bf0573f4f6ca1e7eef9aedd4ed8, State: Initialized, Role: FOLLOWER
I20260925 02:27:05.617369 13201 ts_tablet_manager.cc:1428] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 02:27:05.617353 13202 consensus_queue.cc:260] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4645e2b7358a4d9096d805c4c325ed0c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46603 } } peers { permanent_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33863 } } peers { permanent_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 44901 } }
I20260925 02:27:05.617481 13186 heartbeater.cc:503] Master 127.11.220.254:43655 was elected leader, sending a full tablet report...
I20260925 02:27:05.617434 13200 raft_consensus.cc:348] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4645e2b7358a4d9096d805c4c325ed0c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46603 } } peers { permanent_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33863 } } peers { permanent_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 44901 } }
I20260925 02:27:05.617600 13202 ts_tablet_manager.cc:1428] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:27:05.617602 13200 raft_consensus.cc:374] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:05.617645 13200 raft_consensus.cc:729] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ce88b93d964545a65f0e5c1b7f08fc, State: Initialized, Role: FOLLOWER
I20260925 02:27:05.617756 13200 consensus_queue.cc:260] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4645e2b7358a4d9096d805c4c325ed0c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46603 } } peers { permanent_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33863 } } peers { permanent_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 44901 } }
I20260925 02:27:05.618029 13200 ts_tablet_manager.cc:1428] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:27:05.618042 13050 heartbeater.cc:503] Master 127.11.220.254:43655 was elected leader, sending a full tablet report...
I20260925 02:27:05.618182 12915 heartbeater.cc:503] Master 127.11.220.254:43655 was elected leader, sending a full tablet report...
W20260925 02:27:05.691244 13051 tablet.cc:2369] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:27:05.798017 12916 tablet.cc:2369] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:27:05.830583 13187 tablet.cc:2369] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:05.911296 13207 raft_consensus.cc:482] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:05.911399 13207 raft_consensus.cc:504] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4645e2b7358a4d9096d805c4c325ed0c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46603 } } peers { permanent_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33863 } } peers { permanent_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 44901 } }
I20260925 02:27:05.911736 13207 leader_election.cc:288] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4645e2b7358a4d9096d805c4c325ed0c (127.11.220.195:46603), 01ce88b93d964545a65f0e5c1b7f08fc (127.11.220.194:44901)
I20260925 02:27:05.914530 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.911979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:39753 (local address 127.11.220.195:46603)
0925 02:27:05.912024 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.912026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.912170 (+ 144us) server_negotiation.cc:407] Connection header received
0925 02:27:05.912252 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.912252 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.912268 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.912333 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:05.912526 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.912962 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.913856 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.914198 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.914280 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.914288 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.914435 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.914468 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 02:27:05.914572 13199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.911976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:50889 (local address 127.11.220.194:44901)
0925 02:27:05.912024 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.912026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.912116 (+ 90us) server_negotiation.cc:407] Connection header received
0925 02:27:05.912192 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.912193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.912204 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.912269 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:05.912441 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.912931 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.913844 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.914226 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.914300 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.914308 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.914475 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.914506 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 02:27:05.914588 13209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.911869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:46603 (local address 127.11.220.193:39753)
0925 02:27:05.912006 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:05.912111 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:05.912187 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:05.912339 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:05.912340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:05.912342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:05.912422 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.912424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.913038 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.913040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.913762 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.913764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.914278 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.914279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.914341 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.914345 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:05.914427 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 02:27:05.914461 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":57,"threads_started":1}
I20260925 02:27:05.914602 13210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.911901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:44901 (local address 127.11.220.193:50889)
0925 02:27:05.912043 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:05.912051 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:05.912132 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:05.912275 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:05.912276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:05.912277 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:05.912361 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.912363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.913013 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.913014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.913753 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:05.913755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.914300 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.914301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:05.914391 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.914395 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:05.914467 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:27:05.914505 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":14,"thread_start_us":72,"threads_started":1}
I20260925 02:27:05.914932 13139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897c4b7d11724b3b9314a5c8bb5e47a6" candidate_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4645e2b7358a4d9096d805c4c325ed0c" is_pre_election: true
I20260925 02:27:05.914930 13003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897c4b7d11724b3b9314a5c8bb5e47a6" candidate_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" is_pre_election: true
I20260925 02:27:05.915140 13003 raft_consensus.cc:2454] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5d59bf0573f4f6ca1e7eef9aedd4ed8 in term 0.
I20260925 02:27:05.915135 13139 raft_consensus.cc:2454] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5d59bf0573f4f6ca1e7eef9aedd4ed8 in term 0.
I20260925 02:27:05.915372 12801 leader_election.cc:302] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4645e2b7358a4d9096d805c4c325ed0c, e5d59bf0573f4f6ca1e7eef9aedd4ed8; no voters:
I20260925 02:27:05.915544 13207 raft_consensus.cc:2781] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:05.915608 13207 raft_consensus.cc:482] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:05.915625 13207 raft_consensus.cc:3037] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:05.916344 13207 raft_consensus.cc:504] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4645e2b7358a4d9096d805c4c325ed0c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46603 } } peers { permanent_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33863 } } peers { permanent_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 44901 } }
I20260925 02:27:05.916565 13207 leader_election.cc:288] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [CANDIDATE]: Term 1 election: Requested vote from peers 4645e2b7358a4d9096d805c4c325ed0c (127.11.220.195:46603), 01ce88b93d964545a65f0e5c1b7f08fc (127.11.220.194:44901)
I20260925 02:27:05.916630 13139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897c4b7d11724b3b9314a5c8bb5e47a6" candidate_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4645e2b7358a4d9096d805c4c325ed0c"
I20260925 02:27:05.916698 13139 raft_consensus.cc:3037] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:05.916699 13003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "897c4b7d11724b3b9314a5c8bb5e47a6" candidate_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ce88b93d964545a65f0e5c1b7f08fc"
I20260925 02:27:05.916762 13003 raft_consensus.cc:3037] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:05.917339 13139 raft_consensus.cc:2454] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 4645e2b7358a4d9096d805c4c325ed0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5d59bf0573f4f6ca1e7eef9aedd4ed8 in term 1.
I20260925 02:27:05.917352 13003 raft_consensus.cc:2454] T 897c4b7d11724b3b9314a5c8bb5e47a6 P 01ce88b93d964545a65f0e5c1b7f08fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5d59bf0573f4f6ca1e7eef9aedd4ed8 in term 1.
I20260925 02:27:05.917476 12801 leader_election.cc:302] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4645e2b7358a4d9096d805c4c325ed0c, e5d59bf0573f4f6ca1e7eef9aedd4ed8; no voters:
I20260925 02:27:05.917558 13207 raft_consensus.cc:2781] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:05.917686 13207 raft_consensus.cc:686] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [term 1 LEADER]: Becoming Leader. State: Replica: e5d59bf0573f4f6ca1e7eef9aedd4ed8, State: Running, Role: LEADER
I20260925 02:27:05.917773 13207 consensus_queue.cc:237] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4645e2b7358a4d9096d805c4c325ed0c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46603 } } peers { permanent_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33863 } } peers { permanent_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 44901 } }
I20260925 02:27:05.918527 12720 catalog_manager.cc:5495] T 897c4b7d11724b3b9314a5c8bb5e47a6 P e5d59bf0573f4f6ca1e7eef9aedd4ed8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5d59bf0573f4f6ca1e7eef9aedd4ed8 (127.11.220.193). New cstate: current_term: 1 leader_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4645e2b7358a4d9096d805c4c325ed0c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5d59bf0573f4f6ca1e7eef9aedd4ed8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01ce88b93d964545a65f0e5c1b7f08fc" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 44901 } health_report { overall_health: UNKNOWN } } }
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:27:05.962930 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.956124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58896 (local address 127.11.220.193:33863)
0925 02:27:05.956194 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.956196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.956285 (+ 89us) server_negotiation.cc:407] Connection header received
0925 02:27:05.956358 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.956359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.956367 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.956412 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:05.956967 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.957412 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.958076 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.958276 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.958342 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.960477 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:05.960488 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:05.960489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:05.960495 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:05.961782 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.962140 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.962143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.962144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.962204 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.962399 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.962400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.962401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.962539 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:05.962710 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.962749 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.962823 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:27:05.987150 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:05.980710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.11.220.193:33863)
0925 02:27:05.980748 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:05.980750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:05.980762 (+ 12us) server_negotiation.cc:407] Connection header received
0925 02:27:05.980786 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:05.980787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:05.980803 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:05.980846 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:05.981450 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.981888 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.982492 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:05.982652 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:05.982702 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:05.984744 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:05.984745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:05.984746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:05.984752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:05.985966 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.986306 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.986308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.986308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.986359 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:05.986537 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:05.986538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:05.986538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:05.986628 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:05.986808 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:05.986978 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 02:27:05.987050 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 25 02:27:05 dist-test-slave-dn4n krb5kdc[12677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 02:27:06 dist-test-slave-dn4n krb5kdc[12677](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:27:06.006438 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:06.000478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58922 (local address 127.11.220.193:33863)
0925 02:27:06.000533 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:06.000535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:06.000679 (+ 144us) server_negotiation.cc:407] Connection header received
0925 02:27:06.000705 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:06.000706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:06.000716 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:06.000769 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:06.001401 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:06.001801 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:06.002448 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:06.002599 (+ 151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:06.002641 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:06.004405 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:06.004406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:06.004406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:06.004412 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:06.005703 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:06.005968 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:06.005968 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:06.005968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:06.006010 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:06.006140 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:06.006140 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:06.006141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:06.006207 (+ 66us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:06.006283 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:06.006333 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 02:27:06.006381 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260925 02:27:06.006920 12828 tablet_service.cc:2908] Rejecting scan request for tablet 897c4b7d11724b3b9314a5c8bb5e47a6: Uninitialized: clean time has not yet been initialized
I20260925 02:27:06.007929 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 12785
I20260925 02:27:06.013417 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 12920
I20260925 02:27:06.017781 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 13055
I20260925 02:27:06.021821 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 12690
2026-09-25T02:27:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2092 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:06 dist-test-slave-dn4n krb5kdc[13226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:06 dist-test-slave-dn4n krb5kdc[13226](info): set up 2 sockets
Sep 25 02:27:06 dist-test-slave-dn4n krb5kdc[13226](info): commencing operation
krb5kdc: starting...
W20260925 02:27:06.968667 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:07.072553 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:40359
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:40359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:07.145088 13239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:07.145256 13239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:07.145275 13239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:07.146298 13239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:07.146333 13239 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 02:27:07.146345 13239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:07.146358 13239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:07.146402 13239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:07.146420 13239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:07.147739 13239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:40359
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:40359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:07.147914 13239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:07.148121 13239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:07.152661 13239 server_base.cc:1034] running on GCE node
W20260925 02:27:07.152631 13247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:07.152632 13244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:07.152621 13245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:07.153050 13239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:07.153347 13239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:07.154493 13239 hybrid_clock.cc:648] HybridClock initialized: now 1790303227154487 us; error 33 us; skew 500 ppm
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:07.156596 13239 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:07.156955 13239 webserver.cc:460] Webserver started at http://127.11.220.254:45949/ using document root <none> and password file <none>
I20260925 02:27:07.157177 13239 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:07.157240 13239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:07.157359 13239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:07.158203 13239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "3b326b6cc470489a91ca71a8e55e60d6"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "0fe63775c275fd708f70e1cd8b0bb0a9"
server_key_iv: "9f92afe17243c8c3ce0ee0ff77fa85ba"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.158506 13239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "3b326b6cc470489a91ca71a8e55e60d6"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "0fe63775c275fd708f70e1cd8b0bb0a9"
server_key_iv: "9f92afe17243c8c3ce0ee0ff77fa85ba"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.159576 13239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:07.160209 13254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.160354 13239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:07.160447 13239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "3b326b6cc470489a91ca71a8e55e60d6"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "0fe63775c275fd708f70e1cd8b0bb0a9"
server_key_iv: "9f92afe17243c8c3ce0ee0ff77fa85ba"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.160527 13239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:07.171290 13239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:07.173032 13239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:07.173180 13239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:07.177641 13239 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:40359
I20260925 02:27:07.177695 13317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:40359 every 8 connection(s)
I20260925 02:27:07.177948 13239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:07.178472 13318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:07.180426 13318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6: Bootstrap starting.
I20260925 02:27:07.181006 13318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:07.181295 13318 log.cc:824] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:07.181931 13318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6: No bootstrap required, opened a new log
I20260925 02:27:07.183889 13318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b326b6cc470489a91ca71a8e55e60d6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40359 } }
I20260925 02:27:07.184022 13318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:07.184062 13318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b326b6cc470489a91ca71a8e55e60d6, State: Initialized, Role: FOLLOWER
I20260925 02:27:07.184233 13318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b326b6cc470489a91ca71a8e55e60d6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40359 } }
I20260925 02:27:07.184317 13318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:07.184357 13318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:07.184402 13318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:07.184861 13318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b326b6cc470489a91ca71a8e55e60d6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40359 } }
I20260925 02:27:07.184968 13318 leader_election.cc:302] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b326b6cc470489a91ca71a8e55e60d6; no voters:
I20260925 02:27:07.185129 13318 leader_election.cc:288] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:07.185154 13321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:07.185299 13321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [term 1 LEADER]: Becoming Leader. State: Replica: 3b326b6cc470489a91ca71a8e55e60d6, State: Running, Role: LEADER
I20260925 02:27:07.185346 13318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:07.185379 13321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b326b6cc470489a91ca71a8e55e60d6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40359 } }
I20260925 02:27:07.185739 13323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b326b6cc470489a91ca71a8e55e60d6. Latest consensus state: current_term: 1 leader_uuid: "3b326b6cc470489a91ca71a8e55e60d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b326b6cc470489a91ca71a8e55e60d6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40359 } } }
I20260925 02:27:07.185830 13323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:07.185974 13322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b326b6cc470489a91ca71a8e55e60d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b326b6cc470489a91ca71a8e55e60d6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40359 } } }
I20260925 02:27:07.186038 13322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:07.186143 13330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:07.186522 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 13239
I20260925 02:27:07.186623 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:07.186640 13330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:07.186772 12147 external_mini_cluster.cc:1428] Setting key 25cc1d5fe85fd75aa55acbe7a1219a83
I20260925 02:27:07.188218 13330 catalog_manager.cc:1347] Generated new cluster ID: 2290b29a95394e1388ff46b92b771b7a
I20260925 02:27:07.188265 13330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:07.195757 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.187210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.11.220.254:40359)
0925 02:27:07.187371 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.187380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.187431 (+ 51us) server_negotiation.cc:407] Connection header received
0925 02:27:07.187635 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.187640 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.187718 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.187846 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:07.188559 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.189373 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.190060 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.190467 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.190578 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.192835 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:07.192839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:07.192839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:07.192850 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:07.194349 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.194695 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.194698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.194698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.194791 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.194989 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.194989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.194990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.195226 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:07.195383 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.195471 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.195602 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260925 02:27:07.202809 13330 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:07.203429 13330 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:07.218256 13330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6: Generated new TSK 0
I20260925 02:27:07.218400 13330 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:07.247021 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40359
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:07.319403 13344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:07.319581 13344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:07.319613 13344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:07.320662 13344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:07.320720 13344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:07.320750 13344 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:27:07.320787 13344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:07.322080 13344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.220.254:40359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:07.322325 13344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:07.322528 13344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:07.326953 13344 server_base.cc:1034] running on GCE node
W20260925 02:27:07.326906 13352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:07.326903 13350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:07.326906 13349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:07.327400 13344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:07.327606 13344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:07.328758 13344 hybrid_clock.cc:648] HybridClock initialized: now 1790303227328738 us; error 30 us; skew 500 ppm
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:07.330878 13344 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:07.331238 13344 webserver.cc:460] Webserver started at http://127.11.220.193:39917/ using document root <none> and password file <none>
I20260925 02:27:07.331447 13344 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:07.331507 13344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:07.331642 13344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:07.332630 13344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "a5beac2c783a462f86f6fd656a3cfba4"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "32f489aeb6f14f54d9149e99f79686ca"
server_key_iv: "fc6b35e370b8129a66e60fef07e1d273"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.332957 13344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "a5beac2c783a462f86f6fd656a3cfba4"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "32f489aeb6f14f54d9149e99f79686ca"
server_key_iv: "fc6b35e370b8129a66e60fef07e1d273"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.334159 13344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:07.334770 13359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.334909 13344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:07.334992 13344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "a5beac2c783a462f86f6fd656a3cfba4"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "32f489aeb6f14f54d9149e99f79686ca"
server_key_iv: "fc6b35e370b8129a66e60fef07e1d273"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.335058 13344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:07.345242 13344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:07.346735 13344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:07.346880 13344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:07.347308 13344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:07.347374 13344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.347409 13344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:07.347435 13344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:07.353892 13344 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:40007
I20260925 02:27:07.353943 13475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:40007 every 8 connection(s)
I20260925 02:27:07.354233 13344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:07.356570 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.347718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:51591 (local address 127.11.220.254:40359)
0925 02:27:07.347785 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.347787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.347982 (+ 195us) server_negotiation.cc:407] Connection header received
0925 02:27:07.348341 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.348343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.348358 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.348428 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:07.349244 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.349998 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.351400 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.351762 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.351843 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.354244 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:07.354246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:07.354246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:07.354257 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:07.355498 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.355822 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.355823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.355823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.355868 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.356029 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.356030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.356030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.356115 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:07.356160 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.356369 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.356428 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:27:07.356665 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.347694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40359 (local address 127.11.220.193:51591)
0925 02:27:07.347852 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.347884 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.348252 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.348540 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.348543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.348865 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:07.349131 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.349137 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.350094 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.350095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.351312 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.351314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.351870 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.351871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.352002 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.352548 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:07.352555 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:07.354146 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:07.355569 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.355574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.355575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.355761 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.355913 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.355913 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.355913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.355985 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.356173 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:07.356178 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:07.356270 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.356342 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.356437 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260925 02:27:07.357076 13476 heartbeater.cc:347] Connected to a master server at 127.11.220.254:40359
I20260925 02:27:07.357147 13476 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:07.357340 13476 heartbeater.cc:511] Master 127.11.220.254:40359 requested a full tablet report, sending...
I20260925 02:27:07.357780 13270 ts_manager.cc:194] Registered new tserver with Master: a5beac2c783a462f86f6fd656a3cfba4 (127.11.220.193:40007)
I20260925 02:27:07.358539 13270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:51591
I20260925 02:27:07.361157 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 13344
I20260925 02:27:07.361267 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:07.361435 12147 external_mini_cluster.cc:1428] Setting key 18dea3849cdb657ef33eb4b3ddbcace0
I20260925 02:27:07.362974 13482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.357988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40359 (local address 127.11.220.193:58929)
0925 02:27:07.358092 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.358100 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.358152 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.358265 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.358265 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.358395 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:07.358437 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.358438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.359149 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.359149 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.359623 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.359625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.359902 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.359902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.359947 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.360340 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:07.360345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:07.360653 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:07.362024 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.362025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.362025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.362248 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.362420 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.362420 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.362420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.362462 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.362710 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:07.362711 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:07.362758 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.362810 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.362920 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":59,"threads_started":1}
I20260925 02:27:07.363055 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.358047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:58929 (local address 127.11.220.254:40359)
0925 02:27:07.358082 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.358084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.358142 (+ 58us) server_negotiation.cc:407] Connection header received
0925 02:27:07.358204 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.358205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.358213 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.358261 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:07.358493 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.359059 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.359686 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.359845 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.359886 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.360714 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:07.360715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:07.360715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:07.360720 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:07.361962 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.362298 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.362299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.362299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.362356 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.362496 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.362497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.362497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.362615 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:07.362716 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.362850 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.362937 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:07.382768 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40359
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:07.455062 13485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:07.455255 13485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:07.455286 13485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:07.456307 13485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:07.456365 13485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:07.456394 13485 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:27:07.456449 13485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:07.457922 13485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.220.254:40359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:07.458158 13485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:07.458381 13485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:07.462873 13491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:07.462911 13490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:07.462909 13493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:07.463078 13485 server_base.cc:1034] running on GCE node
I20260925 02:27:07.463264 13485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:07.463471 13485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:07.464629 13485 hybrid_clock.cc:648] HybridClock initialized: now 1790303227464609 us; error 30 us; skew 500 ppm
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:07.466782 13485 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:07.467156 13485 webserver.cc:460] Webserver started at http://127.11.220.194:41707/ using document root <none> and password file <none>
I20260925 02:27:07.467352 13485 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:07.467411 13485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:07.467542 13485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:07.468420 13485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "f2c6fa29942941eea69bb6c9487a633a"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "b087026c5b0d1b816de56d201c7f29f4"
server_key_iv: "8b3c7eaf1f08488afd5fba36aac77f66"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.468809 13485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "f2c6fa29942941eea69bb6c9487a633a"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "b087026c5b0d1b816de56d201c7f29f4"
server_key_iv: "8b3c7eaf1f08488afd5fba36aac77f66"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.469938 13485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:07.470561 13500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.470726 13485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:07.470809 13485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "f2c6fa29942941eea69bb6c9487a633a"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "b087026c5b0d1b816de56d201c7f29f4"
server_key_iv: "8b3c7eaf1f08488afd5fba36aac77f66"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.470886 13485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:07.486241 13485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:07.488258 13485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:07.488358 13485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:07.488811 13485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:07.488871 13485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.488917 13485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:07.488960 13485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:07.495669 13485 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:34781
I20260925 02:27:07.495712 13616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:34781 every 8 connection(s)
I20260925 02:27:07.496037 13485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:07.496639 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 13485
I20260925 02:27:07.496762 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:07.496924 12147 external_mini_cluster.cc:1428] Setting key 9aad2846712731ab47cf470a365503de
I20260925 02:27:07.498795 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.489257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:39187 (local address 127.11.220.254:40359)
0925 02:27:07.489322 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.489325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.489472 (+ 147us) server_negotiation.cc:407] Connection header received
0925 02:27:07.489853 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.489854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.489868 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.489944 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:07.490580 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.491349 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.492926 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.493290 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.493384 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.495825 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:07.495826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:07.495827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:07.495834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:07.497453 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.497959 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.497960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.497960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.498007 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.498252 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.498253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.498253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.498340 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:07.498385 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.498679 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.498734 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:27:07.499066 13509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.489234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40359 (local address 127.11.220.194:39187)
0925 02:27:07.489361 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.489394 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.489766 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.489984 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.489987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.490236 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:07.490485 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.490490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.491447 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.491449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.492847 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.492851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.493412 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.493413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.493534 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.494008 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:07.494019 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:07.495750 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:07.497572 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.497580 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.497582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.497892 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.498065 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.498066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.498066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.498193 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.498396 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:07.498401 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:07.498537 (+ 136us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.498658 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.498823 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":72,"threads_started":1}
I20260925 02:27:07.499502 13617 heartbeater.cc:347] Connected to a master server at 127.11.220.254:40359
I20260925 02:27:07.499573 13617 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:07.499743 13617 heartbeater.cc:511] Master 127.11.220.254:40359 requested a full tablet report, sending...
I20260925 02:27:07.500102 13271 ts_manager.cc:194] Registered new tserver with Master: f2c6fa29942941eea69bb6c9487a633a (127.11.220.194:34781)
I20260925 02:27:07.500615 13271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:39187
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
I20260925 02:27:07.506011 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.500522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40359 (local address 127.11.220.194:58785)
0925 02:27:07.500633 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.500642 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.500725 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.500848 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.500848 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.501043 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:07.501095 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.501097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.502018 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.502019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.502481 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.502483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.502974 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.502975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.503026 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.503373 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:07.503376 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:07.503598 (+ 222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:07.505114 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.505115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.505115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.505258 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.505439 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.505439 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.505439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.505483 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.505787 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:07.505788 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:07.505839 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.505889 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.505954 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":67,"threads_started":1}
I20260925 02:27:07.506045 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.500554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:58785 (local address 127.11.220.254:40359)
0925 02:27:07.500596 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.500598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.500714 (+ 116us) server_negotiation.cc:407] Connection header received
0925 02:27:07.500780 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.500781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.500790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.500842 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:07.501183 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.501931 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.502561 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.502891 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.502977 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.503674 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:07.503675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:07.503675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:07.503682 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:07.505043 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.505327 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.505328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.505329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.505381 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.505531 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.505532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.505532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.505721 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:07.505795 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.505915 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.505982 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:07.520987 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40359
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:07.594736 13626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:07.594905 13626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:07.594928 13626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:07.596019 13626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:07.596048 13626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:07.596067 13626 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:27:07.596083 13626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:07.597422 13626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.220.254:40359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:07.597585 13626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:07.597733 13626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:07.602353 13634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:07.602375 13631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:07.602350 13632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:07.602425 13626 server_base.cc:1034] running on GCE node
I20260925 02:27:07.602744 13626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:07.602981 13626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:07.604127 13626 hybrid_clock.cc:648] HybridClock initialized: now 1790303227604115 us; error 42 us; skew 500 ppm
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:07.606357 13626 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:07.606706 13626 webserver.cc:460] Webserver started at http://127.11.220.195:33841/ using document root <none> and password file <none>
I20260925 02:27:07.606885 13626 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:07.606922 13626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:07.607002 13626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:07.607878 13626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "cc765a2a9f514edba4c54805c53b41a6"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "20764d8e1d5b3990ea52005a568bb49f"
server_key_iv: "9aaa51c2da00ce43905b7d97c44bfde6"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.608129 13626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "cc765a2a9f514edba4c54805c53b41a6"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "20764d8e1d5b3990ea52005a568bb49f"
server_key_iv: "9aaa51c2da00ce43905b7d97c44bfde6"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.609325 13626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:27:07.610091 13641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.610296 13626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:07.610373 13626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "cc765a2a9f514edba4c54805c53b41a6"
format_stamp: "Formatted at 2026-09-25 02:27:07 on dist-test-slave-dn4n"
server_key: "20764d8e1d5b3990ea52005a568bb49f"
server_key_iv: "9aaa51c2da00ce43905b7d97c44bfde6"
server_key_version: "encryptionkey@0"
I20260925 02:27:07.610426 13626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:07.621345 13626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:07.622852 13626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:07.622951 13626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:07.623320 13626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:07.623355 13626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.623374 13626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:07.623430 13626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:07.629951 13626 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:33947
I20260925 02:27:07.630012 13758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:33947 every 8 connection(s)
I20260925 02:27:07.630259 13626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:07.633131 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.623786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:37199 (local address 127.11.220.254:40359)
0925 02:27:07.623859 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.623862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.623988 (+ 126us) server_negotiation.cc:407] Connection header received
0925 02:27:07.624387 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.624389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.624405 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.624504 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:07.625217 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.625953 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.627557 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.627913 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.627983 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.630431 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:07.630432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:07.630433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:07.630439 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:07.631762 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.632178 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.632180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.632180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.632240 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.632503 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.632504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.632504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.632631 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:07.632718 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.632973 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.633046 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:27:07.633285 13650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.623741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40359 (local address 127.11.220.195:37199)
0925 02:27:07.623870 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.623906 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.624300 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.624581 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.624584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.624897 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:07.625138 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.625143 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.626063 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.626064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.627458 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.627461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.628030 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.628031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.628177 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.628670 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:07.628680 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:07.630354 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:07.631876 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.631884 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.631885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.632092 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.632306 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.632307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.632307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.632411 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.632712 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:07.632719 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:07.632843 (+ 124us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.632950 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.633079 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":71,"threads_started":1}
I20260925 02:27:07.633739 13759 heartbeater.cc:347] Connected to a master server at 127.11.220.254:40359
I20260925 02:27:07.633828 13759 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:07.633975 13759 heartbeater.cc:511] Master 127.11.220.254:40359 requested a full tablet report, sending...
I20260925 02:27:07.634331 13271 ts_manager.cc:194] Registered new tserver with Master: cc765a2a9f514edba4c54805c53b41a6 (127.11.220.195:33947)
I20260925 02:27:07.634651 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 13626
I20260925 02:27:07.634727 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:07.634800 13271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:37199
I20260925 02:27:07.634887 12147 external_mini_cluster.cc:1428] Setting key 0a5c67a4377113bac0782a707ca19eb5
I20260925 02:27:07.636191 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:27:07.640206 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.634651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:36217 (local address 127.11.220.254:40359)
0925 02:27:07.634695 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.634697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.634824 (+ 127us) server_negotiation.cc:407] Connection header received
0925 02:27:07.634905 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.634906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.634915 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.634958 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:07.635332 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.635815 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.636612 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.636916 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.636985 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.637848 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:07.637849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:07.637850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:07.637857 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:07.639202 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.639569 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.639570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.639570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.639615 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.639802 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.639803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.639803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.639887 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:07.639932 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.640095 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.640153 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 02:27:07.640331 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.634605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40359 (local address 127.11.220.195:36217)
0925 02:27:07.634776 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.634785 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.634836 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.634992 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.634993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.635181 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:07.635235 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.635237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.635906 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.635907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.636532 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.636538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.636990 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.636990 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.637042 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.637486 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:07.637490 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:07.637769 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:07.639279 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.639280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.639280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.639500 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.639677 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.639678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.639678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.639745 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.639945 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:07.639945 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:07.639992 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.640081 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.640197 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":115,"threads_started":1}
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:07.657825 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.651719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54754 (local address 127.11.220.254:40359)
0925 02:27:07.651782 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.651784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.651823 (+ 39us) server_negotiation.cc:407] Connection header received
0925 02:27:07.651898 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.651899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.651911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.651968 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:07.652509 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.652950 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.653602 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.653765 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.653836 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.655733 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:07.655734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:07.655734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:07.655740 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:07.656842 (+ 1102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.657182 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.657183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.657183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.657227 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.657374 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.657375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.657375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.657456 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:07.657523 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.657649 (+ 126us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.657708 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:27:07.659250 13271 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:07.660933 13271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:07.666724 13773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.662849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:40007 (local address 127.0.0.1:42868)
0925 02:27:07.662948 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.662981 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.663034 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.663268 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.663271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.663273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:07.663331 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.663333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.664726 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.664727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.665578 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.665580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.666428 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.666429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.666518 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.666530 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.666604 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.666649 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":52,"thread_start_us":47,"threads_started":1}
I20260925 02:27:07.666762 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.662926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.11.220.193:40007)
0925 02:27:07.663089 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.663093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.663112 (+ 19us) server_negotiation.cc:407] Connection header received
0925 02:27:07.663149 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.663151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.663165 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.663247 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:07.663445 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.664622 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.665669 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.666350 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.666420 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.666466 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.666629 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.666669 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260925 02:27:07.667158 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.663028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:34781 (local address 127.0.0.1:44910)
0925 02:27:07.663423 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.663434 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.663484 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.663747 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.663747 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.663750 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:07.663794 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.663795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.664993 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.664993 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.666078 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.666079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.666945 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.666946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.667010 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.667014 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.667057 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.667093 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":43,"threads_started":1}
I20260925 02:27:07.667294 13775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.663146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:33947 (local address 127.0.0.1:56084)
0925 02:27:07.663552 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.663560 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.663606 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.663867 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.663868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.663869 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:07.663915 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.663917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.664863 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.664864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.665889 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.665891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.667037 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.667038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.667102 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.667107 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.667182 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.667221 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":86,"threads_started":1}
I20260925 02:27:07.667317 13777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.663242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56084 (local address 127.11.220.195:33947)
0925 02:27:07.663409 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.663415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.663605 (+ 190us) server_negotiation.cc:407] Connection header received
0925 02:27:07.663714 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.663716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.663732 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.663784 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:07.663998 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.664774 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.665958 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.666883 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.666951 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.667004 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.667201 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.667235 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260925 02:27:07.667423 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.663386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.11.220.194:34781)
0925 02:27:07.663543 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.663547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.663556 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:07.663591 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.663593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.663607 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.663680 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:07.663867 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.664924 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.666147 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.666792 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.666860 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.666911 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.667254 (+ 343us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.667293 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260925 02:27:07.667511 13409 tablet_service.cc:1467] Processing CreateTablet for tablet ce3502ad975544409acc23312c66c1b0 (DEFAULT_TABLE table=test-table [id=2d458e7f5c934bfcb02314de4a9e077a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:07.667776 13409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3502ad975544409acc23312c66c1b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:07.669078 13692 tablet_service.cc:1467] Processing CreateTablet for tablet ce3502ad975544409acc23312c66c1b0 (DEFAULT_TABLE table=test-table [id=2d458e7f5c934bfcb02314de4a9e077a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:07.669112 13550 tablet_service.cc:1467] Processing CreateTablet for tablet ce3502ad975544409acc23312c66c1b0 (DEFAULT_TABLE table=test-table [id=2d458e7f5c934bfcb02314de4a9e077a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:07.669384 13692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3502ad975544409acc23312c66c1b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:07.669459 13550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3502ad975544409acc23312c66c1b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:07.671365 13779 tablet_bootstrap.cc:492] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: Bootstrap starting.
I20260925 02:27:07.672073 13779 tablet_bootstrap.cc:654] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:07.672290 13780 tablet_bootstrap.cc:492] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: Bootstrap starting.
I20260925 02:27:07.672432 13779 log.cc:824] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:07.673015 13782 tablet_bootstrap.cc:492] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: Bootstrap starting.
I20260925 02:27:07.673288 13780 tablet_bootstrap.cc:654] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:07.673295 13779 tablet_bootstrap.cc:492] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: No bootstrap required, opened a new log
I20260925 02:27:07.673357 13779 ts_tablet_manager.cc:1397] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:27:07.673593 13780 log.cc:824] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:07.673642 13782 tablet_bootstrap.cc:654] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:07.673959 13782 log.cc:824] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:07.674356 13780 tablet_bootstrap.cc:492] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: No bootstrap required, opened a new log
I20260925 02:27:07.674423 13780 ts_tablet_manager.cc:1397] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:27:07.675122 13779 raft_consensus.cc:348] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.675246 13779 raft_consensus.cc:374] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:07.675266 13779 raft_consensus.cc:729] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5beac2c783a462f86f6fd656a3cfba4, State: Initialized, Role: FOLLOWER
I20260925 02:27:07.675375 13779 consensus_queue.cc:260] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.675624 13779 ts_tablet_manager.cc:1428] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:27:07.675637 13476 heartbeater.cc:503] Master 127.11.220.254:40359 was elected leader, sending a full tablet report...
I20260925 02:27:07.676019 13780 raft_consensus.cc:348] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.676151 13780 raft_consensus.cc:374] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:07.676175 13780 raft_consensus.cc:729] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc765a2a9f514edba4c54805c53b41a6, State: Initialized, Role: FOLLOWER
I20260925 02:27:07.676301 13780 consensus_queue.cc:260] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.676571 13780 ts_tablet_manager.cc:1428] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:27:07.676596 13759 heartbeater.cc:503] Master 127.11.220.254:40359 was elected leader, sending a full tablet report...
I20260925 02:27:07.677023 13782 tablet_bootstrap.cc:492] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: No bootstrap required, opened a new log
I20260925 02:27:07.677098 13782 ts_tablet_manager.cc:1397] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 02:27:07.678337 13782 raft_consensus.cc:348] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.678490 13782 raft_consensus.cc:374] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:07.678536 13782 raft_consensus.cc:729] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c6fa29942941eea69bb6c9487a633a, State: Initialized, Role: FOLLOWER
I20260925 02:27:07.678638 13782 consensus_queue.cc:260] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.678866 13782 ts_tablet_manager.cc:1428] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:27:07.678920 13617 heartbeater.cc:503] Master 127.11.220.254:40359 was elected leader, sending a full tablet report...
I20260925 02:27:07.721167 13787 raft_consensus.cc:482] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:07.721271 13787 raft_consensus.cc:504] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.721670 13787 leader_election.cc:288] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5beac2c783a462f86f6fd656a3cfba4 (127.11.220.193:40007), cc765a2a9f514edba4c54805c53b41a6 (127.11.220.195:33947)
I20260925 02:27:07.725512 13777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.722047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:55489 (local address 127.11.220.195:33947)
0925 02:27:07.722105 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.722106 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.722172 (+ 66us) server_negotiation.cc:407] Connection header received
0925 02:27:07.722206 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.722207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.722215 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.722286 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:07.722430 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.723131 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.724605 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.725161 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.725236 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.725244 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.725402 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.725436 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 02:27:07.725512 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.721924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:55615 (local address 127.11.220.193:40007)
0925 02:27:07.722015 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.722017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.722025 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:07.722057 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.722058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.722080 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.722140 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:07.722289 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.723102 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.724610 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.725159 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.725236 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.725244 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.725403 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.725438 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 02:27:07.725535 13509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.721825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:40007 (local address 127.11.220.194:55615)
0925 02:27:07.721877 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.721889 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.721932 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.722147 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.722148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.722149 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:07.722223 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.722225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.723199 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.723200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.724516 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.724519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.725241 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.725242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.725329 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.725332 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.725384 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.725423 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":46}
I20260925 02:27:07.725545 13788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.721952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:33947 (local address 127.11.220.194:55489)
0925 02:27:07.722110 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.722119 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.722167 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.722289 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.722289 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.722290 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:07.722368 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.722368 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.723199 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.723200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.724516 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.724519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.725241 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.725242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.725319 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.725322 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.725380 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.725423 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":62,"thread_start_us":109,"threads_started":1}
I20260925 02:27:07.725863 13712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3502ad975544409acc23312c66c1b0" candidate_uuid: "f2c6fa29942941eea69bb6c9487a633a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc765a2a9f514edba4c54805c53b41a6" is_pre_election: true
I20260925 02:27:07.725863 13429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3502ad975544409acc23312c66c1b0" candidate_uuid: "f2c6fa29942941eea69bb6c9487a633a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5beac2c783a462f86f6fd656a3cfba4" is_pre_election: true
I20260925 02:27:07.725997 13429 raft_consensus.cc:2454] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c6fa29942941eea69bb6c9487a633a in term 0.
I20260925 02:27:07.725997 13712 raft_consensus.cc:2454] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c6fa29942941eea69bb6c9487a633a in term 0.
I20260925 02:27:07.726198 13502 leader_election.cc:302] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc765a2a9f514edba4c54805c53b41a6, f2c6fa29942941eea69bb6c9487a633a; no voters:
I20260925 02:27:07.726297 13787 raft_consensus.cc:2781] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:07.726346 13787 raft_consensus.cc:482] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:07.726379 13787 raft_consensus.cc:3037] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:07.727221 13787 raft_consensus.cc:504] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.727337 13787 leader_election.cc:288] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [CANDIDATE]: Term 1 election: Requested vote from peers a5beac2c783a462f86f6fd656a3cfba4 (127.11.220.193:40007), cc765a2a9f514edba4c54805c53b41a6 (127.11.220.195:33947)
I20260925 02:27:07.727519 13712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3502ad975544409acc23312c66c1b0" candidate_uuid: "f2c6fa29942941eea69bb6c9487a633a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc765a2a9f514edba4c54805c53b41a6"
I20260925 02:27:07.727535 13429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3502ad975544409acc23312c66c1b0" candidate_uuid: "f2c6fa29942941eea69bb6c9487a633a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5beac2c783a462f86f6fd656a3cfba4"
I20260925 02:27:07.727591 13712 raft_consensus.cc:3037] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:07.727625 13429 raft_consensus.cc:3037] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:07.728230 13712 raft_consensus.cc:2454] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c6fa29942941eea69bb6c9487a633a in term 1.
I20260925 02:27:07.728379 13502 leader_election.cc:302] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc765a2a9f514edba4c54805c53b41a6, f2c6fa29942941eea69bb6c9487a633a; no voters:
I20260925 02:27:07.728394 13429 raft_consensus.cc:2454] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c6fa29942941eea69bb6c9487a633a in term 1.
I20260925 02:27:07.728482 13787 raft_consensus.cc:2781] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:07.728605 13787 raft_consensus.cc:686] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 1 LEADER]: Becoming Leader. State: Replica: f2c6fa29942941eea69bb6c9487a633a, State: Running, Role: LEADER
I20260925 02:27:07.728680 13787 consensus_queue.cc:237] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.729413 13271 catalog_manager.cc:5495] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a reported cstate change: term changed from 0 to 1, leader changed from <none> to f2c6fa29942941eea69bb6c9487a633a (127.11.220.194). New cstate: current_term: 1 leader_uuid: "f2c6fa29942941eea69bb6c9487a633a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260925 02:27:07.739764 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.737365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44918 (local address 127.11.220.194:34781)
0925 02:27:07.737426 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.737428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.737438 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:27:07.737528 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.737529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.737543 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.737586 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:07.737725 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.738200 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.738963 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.739156 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.739230 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.739443 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:07.739533 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:07.739627 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.739654 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.739676 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:27:07.741358 13429 raft_consensus.cc:1261] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4 [term 1 FOLLOWER]: Refusing update from remote peer f2c6fa29942941eea69bb6c9487a633a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:07.741385 13712 raft_consensus.cc:1261] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6 [term 1 FOLLOWER]: Refusing update from remote peer f2c6fa29942941eea69bb6c9487a633a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:07.741677 13787 consensus_queue.cc:1035] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:07.741793 13789 consensus_queue.cc:1035] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:07.743850 13793 mvcc.cc:205] Tried to move back new op lower bound from 7333082020826451968 to 7333082020777226240. Current Snapshot: MvccSnapshot[applied={T|T < 7333082020826451968}]
I20260925 02:27:07.744328 13794 mvcc.cc:205] Tried to move back new op lower bound from 7333082020826451968 to 7333082020777226240. Current Snapshot: MvccSnapshot[applied={T|T < 7333082020826451968 or (T in {7333082020826451968})}]
I20260925 02:27:07.744242 13792 mvcc.cc:205] Tried to move back new op lower bound from 7333082020826451968 to 7333082020777226240. Current Snapshot: MvccSnapshot[applied={T|T < 7333082020826451968 or (T in {7333082020826451968})}]
I20260925 02:27:07.746654 13271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:07.746729 13271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:07.747393 13271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6: Sending DeleteTablet for 3 replicas of tablet ce3502ad975544409acc23312c66c1b0
I20260925 02:27:07.747653 13409 tablet_service.cc:1514] Processing DeleteTablet for tablet ce3502ad975544409acc23312c66c1b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:07 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:42868
I20260925 02:27:07.747750 13550 tablet_service.cc:1514] Processing DeleteTablet for tablet ce3502ad975544409acc23312c66c1b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:07 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:44910
I20260925 02:27:07.747841 13801 tablet_replica.cc:330] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: stopping tablet replica
I20260925 02:27:07.747943 13801 raft_consensus.cc:2229] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:07.747941 13692 tablet_service.cc:1514] Processing DeleteTablet for tablet ce3502ad975544409acc23312c66c1b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:07 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:56084
I20260925 02:27:07.748027 13802 tablet_replica.cc:330] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: stopping tablet replica
I20260925 02:27:07.748054 13801 raft_consensus.cc:2258] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:07.748157 13802 raft_consensus.cc:2229] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:07.748294 13802 raft_consensus.cc:2258] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:07.748296 13803 tablet_replica.cc:330] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: stopping tablet replica
I20260925 02:27:07.748302 13271 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 02:27:07.748373 13803 raft_consensus.cc:2229] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 02:27:07.748437 13271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:07.748922 13801 ts_tablet_manager.cc:1905] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:07.749119 13802 ts_tablet_manager.cc:1905] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:07.750180 13409 tablet_service.cc:1467] Processing CreateTablet for tablet e6d64ebcb1da4ec99a4e270ecf24c5ab (DEFAULT_TABLE table=test-table [id=29710a174de44a35aadc7b1955b77daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:07.750373 13409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d64ebcb1da4ec99a4e270ecf24c5ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:07.750962 13802 ts_tablet_manager.cc:1918] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:07.751071 13801 ts_tablet_manager.cc:1918] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:07.751080 13802 log.cc:1196] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/ce3502ad975544409acc23312c66c1b0
I20260925 02:27:07.751144 13801 log.cc:1196] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/ce3502ad975544409acc23312c66c1b0
I20260925 02:27:07.751376 13801 ts_tablet_manager.cc:1939] T ce3502ad975544409acc23312c66c1b0 P a5beac2c783a462f86f6fd656a3cfba4: Deleting consensus metadata
I20260925 02:27:07.751605 13802 ts_tablet_manager.cc:1939] T ce3502ad975544409acc23312c66c1b0 P f2c6fa29942941eea69bb6c9487a633a: Deleting consensus metadata
I20260925 02:27:07.751684 13550 tablet_service.cc:1467] Processing CreateTablet for tablet e6d64ebcb1da4ec99a4e270ecf24c5ab (DEFAULT_TABLE table=test-table [id=29710a174de44a35aadc7b1955b77daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:07.751789 13550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d64ebcb1da4ec99a4e270ecf24c5ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:07.752138 13256 catalog_manager.cc:4841] TS f2c6fa29942941eea69bb6c9487a633a (127.11.220.194:34781): tablet ce3502ad975544409acc23312c66c1b0 (table test-table [id=2d458e7f5c934bfcb02314de4a9e077a]) successfully deleted
I20260925 02:27:07.752564 13779 tablet_bootstrap.cc:492] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: Bootstrap starting.
I20260925 02:27:07.752640 13782 tablet_bootstrap.cc:492] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a: Bootstrap starting.
I20260925 02:27:07.752967 13779 tablet_bootstrap.cc:654] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:07.753000 13782 tablet_bootstrap.cc:654] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:07.753293 13692 tablet_service.cc:1467] Processing CreateTablet for tablet e6d64ebcb1da4ec99a4e270ecf24c5ab (DEFAULT_TABLE table=test-table [id=29710a174de44a35aadc7b1955b77daf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:07.753391 13692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d64ebcb1da4ec99a4e270ecf24c5ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:07.753548 13782 tablet_bootstrap.cc:492] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a: No bootstrap required, opened a new log
I20260925 02:27:07.753579 13782 ts_tablet_manager.cc:1397] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:07.753679 13782 raft_consensus.cc:348] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.753719 13782 raft_consensus.cc:374] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:07.753732 13782 raft_consensus.cc:729] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c6fa29942941eea69bb6c9487a633a, State: Initialized, Role: FOLLOWER
I20260925 02:27:07.753791 13782 consensus_queue.cc:260] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.753871 13782 ts_tablet_manager.cc:1428] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.754079 13779 tablet_bootstrap.cc:492] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: No bootstrap required, opened a new log
I20260925 02:27:07.754141 13779 ts_tablet_manager.cc:1397] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 02:27:07.754280 13779 raft_consensus.cc:348] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.754348 13779 raft_consensus.cc:374] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:07.754367 13779 raft_consensus.cc:729] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5beac2c783a462f86f6fd656a3cfba4, State: Initialized, Role: FOLLOWER
I20260925 02:27:07.754400 13779 consensus_queue.cc:260] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.754484 13779 ts_tablet_manager.cc:1428] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:07.754652 13780 tablet_bootstrap.cc:492] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: Bootstrap starting.
I20260925 02:27:07.754930 13780 tablet_bootstrap.cc:654] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:07.755103 13256 catalog_manager.cc:4841] TS a5beac2c783a462f86f6fd656a3cfba4 (127.11.220.193:40007): tablet ce3502ad975544409acc23312c66c1b0 (table test-table [id=2d458e7f5c934bfcb02314de4a9e077a]) successfully deleted
I20260925 02:27:07.755420 13780 tablet_bootstrap.cc:492] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: No bootstrap required, opened a new log
I20260925 02:27:07.755467 13780 ts_tablet_manager.cc:1397] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 02:27:07.755604 13780 raft_consensus.cc:348] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.755688 13803 raft_consensus.cc:2258] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:07.755702 13780 raft_consensus.cc:374] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:07.755846 13780 raft_consensus.cc:729] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc765a2a9f514edba4c54805c53b41a6, State: Initialized, Role: FOLLOWER
I20260925 02:27:07.755882 13780 consensus_queue.cc:260] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.755972 13780 ts_tablet_manager.cc:1428] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.756402 13803 ts_tablet_manager.cc:1905] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:07.758078 13803 ts_tablet_manager.cc:1918] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:07.758149 13803 log.cc:1196] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/ce3502ad975544409acc23312c66c1b0
I20260925 02:27:07.758384 13803 ts_tablet_manager.cc:1939] T ce3502ad975544409acc23312c66c1b0 P cc765a2a9f514edba4c54805c53b41a6: Deleting consensus metadata
I20260925 02:27:07.758742 13256 catalog_manager.cc:4841] TS cc765a2a9f514edba4c54805c53b41a6 (127.11.220.195:33947): tablet ce3502ad975544409acc23312c66c1b0 (table test-table [id=2d458e7f5c934bfcb02314de4a9e077a]) successfully deleted
W20260925 02:27:07.855134 13477 tablet.cc:2369] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:07.870023 13785 raft_consensus.cc:482] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:07.870095 13785 raft_consensus.cc:504] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.870335 13785 leader_election.cc:288] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2c6fa29942941eea69bb6c9487a633a (127.11.220.194:34781), cc765a2a9f514edba4c54805c53b41a6 (127.11.220.195:33947)
I20260925 02:27:07.873260 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.870598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:38759 (local address 127.11.220.194:34781)
0925 02:27:07.870661 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.870663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.870709 (+ 46us) server_negotiation.cc:407] Connection header received
0925 02:27:07.870774 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.870775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.870786 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.870858 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:07.871019 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.871460 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.872431 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.872873 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.872962 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.872970 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.873123 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.873159 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 02:27:07.873282 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.870542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:34781 (local address 127.11.220.193:38759)
0925 02:27:07.870640 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.870652 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.870712 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.870876 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.870877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.870880 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:07.870952 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.870954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.871536 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.871537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.872326 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.872328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.872962 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.872963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.873038 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.873042 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.873115 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.873156 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":68,"threads_started":1}
I20260925 02:27:07.873281 13777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.870815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:58565 (local address 127.11.220.195:33947)
0925 02:27:07.870863 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.870865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.870873 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:07.870918 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.870919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.870927 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.870976 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:07.871122 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.871538 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.872492 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.872895 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.872983 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.872992 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.873155 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.873204 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 02:27:07.873299 13808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.870712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:33947 (local address 127.11.220.193:58565)
0925 02:27:07.870819 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.870826 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.870865 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.870979 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.870980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.870982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:07.871042 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.871043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.871621 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.871622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.872367 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.872369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.872985 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.872986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.873069 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.873079 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.873146 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.873186 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":36,"threads_started":1}
I20260925 02:27:07.873556 13570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d64ebcb1da4ec99a4e270ecf24c5ab" candidate_uuid: "a5beac2c783a462f86f6fd656a3cfba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c6fa29942941eea69bb6c9487a633a" is_pre_election: true
I20260925 02:27:07.873672 13570 raft_consensus.cc:2454] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5beac2c783a462f86f6fd656a3cfba4 in term 0.
I20260925 02:27:07.873641 13712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d64ebcb1da4ec99a4e270ecf24c5ab" candidate_uuid: "a5beac2c783a462f86f6fd656a3cfba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc765a2a9f514edba4c54805c53b41a6" is_pre_election: true
I20260925 02:27:07.873735 13712 raft_consensus.cc:2454] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5beac2c783a462f86f6fd656a3cfba4 in term 0.
I20260925 02:27:07.873864 13361 leader_election.cc:302] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5beac2c783a462f86f6fd656a3cfba4, f2c6fa29942941eea69bb6c9487a633a; no voters:
I20260925 02:27:07.873993 13785 raft_consensus.cc:2781] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:07.874034 13785 raft_consensus.cc:482] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:07.874061 13785 raft_consensus.cc:3037] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:07.874720 13785 raft_consensus.cc:504] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.874847 13785 leader_election.cc:288] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [CANDIDATE]: Term 1 election: Requested vote from peers f2c6fa29942941eea69bb6c9487a633a (127.11.220.194:34781), cc765a2a9f514edba4c54805c53b41a6 (127.11.220.195:33947)
I20260925 02:27:07.874998 13570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d64ebcb1da4ec99a4e270ecf24c5ab" candidate_uuid: "a5beac2c783a462f86f6fd656a3cfba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2c6fa29942941eea69bb6c9487a633a"
I20260925 02:27:07.875023 13712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d64ebcb1da4ec99a4e270ecf24c5ab" candidate_uuid: "a5beac2c783a462f86f6fd656a3cfba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc765a2a9f514edba4c54805c53b41a6"
I20260925 02:27:07.875087 13712 raft_consensus.cc:3037] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:07.875088 13570 raft_consensus.cc:3037] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:07.875684 13712 raft_consensus.cc:2454] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5beac2c783a462f86f6fd656a3cfba4 in term 1.
I20260925 02:27:07.875689 13570 raft_consensus.cc:2454] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5beac2c783a462f86f6fd656a3cfba4 in term 1.
I20260925 02:27:07.875839 13361 leader_election.cc:302] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5beac2c783a462f86f6fd656a3cfba4, cc765a2a9f514edba4c54805c53b41a6; no voters:
I20260925 02:27:07.875954 13785 raft_consensus.cc:2781] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:07.876082 13785 raft_consensus.cc:686] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 1 LEADER]: Becoming Leader. State: Replica: a5beac2c783a462f86f6fd656a3cfba4, State: Running, Role: LEADER
I20260925 02:27:07.876166 13785 consensus_queue.cc:237] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.876822 13271 catalog_manager.cc:5495] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a5beac2c783a462f86f6fd656a3cfba4 (127.11.220.193). New cstate: current_term: 1 leader_uuid: "a5beac2c783a462f86f6fd656a3cfba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
W20260925 02:27:07.880720 13760 tablet.cc:2369] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 02:27:07 dist-test-slave-dn4n krb5kdc[13226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:07.939052 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.933109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40359 (local address 127.0.0.1:54770)
0925 02:27:07.933265 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:07.933274 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:07.933334 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:07.933443 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:07.933444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:07.933799 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:07.933844 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.933846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.934396 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.934396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.934906 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:07.934907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.935176 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.935176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:07.935221 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.935624 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:07.935628 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:07.936863 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:07.938004 (+ 1141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.938010 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.938012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.938179 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.938367 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:07.938368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:07.938368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:07.938421 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:07.938650 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:07.938657 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:07.938698 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 02:27:07.938778 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 02:27:07.938924 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260925 02:27:07.939059 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:07.933174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54770 (local address 127.11.220.254:40359)
0925 02:27:07.933253 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:07.933255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:07.933332 (+ 77us) server_negotiation.cc:407] Connection header received
0925 02:27:07.933374 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:07.933375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:07.933383 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:07.933431 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:07.933908 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.934339 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.934963 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:07.935122 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:07.935170 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:07.936945 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:07.936946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:07.936947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:07.936952 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:07.937926 (+ 974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.938255 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.938256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.938257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.938302 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:07.938498 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:07.938498 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:07.938499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:07.938589 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:07.938659 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:07.938809 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 02:27:07.938986 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":49}
I20260925 02:27:07.941006 13271 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:54770:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 02:27:07.941241 13271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:07.942657 13409 tablet_service.cc:1467] Processing CreateTablet for tablet 90ecf291895344d5bc66930a40a66793 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5c64e82184141cc9b85fdc4b1e6e5cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:27:07.942706 13550 tablet_service.cc:1467] Processing CreateTablet for tablet 90ecf291895344d5bc66930a40a66793 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5c64e82184141cc9b85fdc4b1e6e5cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:27:07.942714 13692 tablet_service.cc:1467] Processing CreateTablet for tablet 90ecf291895344d5bc66930a40a66793 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5c64e82184141cc9b85fdc4b1e6e5cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:27:07.942826 13409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90ecf291895344d5bc66930a40a66793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:07.942847 13550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90ecf291895344d5bc66930a40a66793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:07.942847 13692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90ecf291895344d5bc66930a40a66793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:07.944108 13779 tablet_bootstrap.cc:492] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: Bootstrap starting.
I20260925 02:27:07.944198 13782 tablet_bootstrap.cc:492] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: Bootstrap starting.
I20260925 02:27:07.944536 13782 tablet_bootstrap.cc:654] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:07.944536 13779 tablet_bootstrap.cc:654] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:07.944947 13780 tablet_bootstrap.cc:492] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: Bootstrap starting.
I20260925 02:27:07.945120 13782 tablet_bootstrap.cc:492] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: No bootstrap required, opened a new log
I20260925 02:27:07.945183 13782 ts_tablet_manager.cc:1397] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:07.945317 13779 tablet_bootstrap.cc:492] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: No bootstrap required, opened a new log
I20260925 02:27:07.945334 13780 tablet_bootstrap.cc:654] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:07.945300 13782 raft_consensus.cc:348] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.945355 13779 ts_tablet_manager.cc:1397] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:07.945365 13782 raft_consensus.cc:374] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:07.945387 13782 raft_consensus.cc:729] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2c6fa29942941eea69bb6c9487a633a, State: Initialized, Role: FOLLOWER
I20260925 02:27:07.945446 13782 consensus_queue.cc:260] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.945534 13782 ts_tablet_manager.cc:1428] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.945497 13779 raft_consensus.cc:348] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.945564 13779 raft_consensus.cc:374] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:07.945588 13779 raft_consensus.cc:729] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5beac2c783a462f86f6fd656a3cfba4, State: Initialized, Role: FOLLOWER
I20260925 02:27:07.945565 13789 tablet_replica.cc:439] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } } }
I20260925 02:27:07.945632 13779 consensus_queue.cc:260] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.945657 13789 tablet_replica.cc:442] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:07.945717 13779 ts_tablet_manager.cc:1428] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.945765 13785 tablet_replica.cc:439] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } } }
I20260925 02:27:07.945820 13785 tablet_replica.cc:442] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:07.946241 13780 tablet_bootstrap.cc:492] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: No bootstrap required, opened a new log
I20260925 02:27:07.946311 13780 ts_tablet_manager.cc:1397] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 02:27:07.946491 13780 raft_consensus.cc:348] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.946599 13780 raft_consensus.cc:374] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:07.946620 13780 raft_consensus.cc:729] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc765a2a9f514edba4c54805c53b41a6, State: Initialized, Role: FOLLOWER
I20260925 02:27:07.946668 13780 consensus_queue.cc:260] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:07.946771 13780 ts_tablet_manager.cc:1428] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:07.946844 13786 tablet_replica.cc:439] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } } }
I20260925 02:27:07.946947 13786 tablet_replica.cc:442] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 02:27:07.997192 13618 tablet.cc:2369] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:08.114279 13789 raft_consensus.cc:482] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:08.114388 13789 raft_consensus.cc:504] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:08.114560 13789 leader_election.cc:288] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5beac2c783a462f86f6fd656a3cfba4 (127.11.220.193:40007), cc765a2a9f514edba4c54805c53b41a6 (127.11.220.195:33947)
I20260925 02:27:08.114800 13429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ecf291895344d5bc66930a40a66793" candidate_uuid: "f2c6fa29942941eea69bb6c9487a633a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5beac2c783a462f86f6fd656a3cfba4" is_pre_election: true
I20260925 02:27:08.114873 13712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ecf291895344d5bc66930a40a66793" candidate_uuid: "f2c6fa29942941eea69bb6c9487a633a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc765a2a9f514edba4c54805c53b41a6" is_pre_election: true
I20260925 02:27:08.114929 13429 raft_consensus.cc:2454] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c6fa29942941eea69bb6c9487a633a in term 0.
I20260925 02:27:08.114979 13712 raft_consensus.cc:2454] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2c6fa29942941eea69bb6c9487a633a in term 0.
I20260925 02:27:08.115098 13502 leader_election.cc:302] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5beac2c783a462f86f6fd656a3cfba4, f2c6fa29942941eea69bb6c9487a633a; no voters:
I20260925 02:27:08.115203 13789 raft_consensus.cc:2781] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:08.115243 13789 raft_consensus.cc:482] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:08.115265 13789 raft_consensus.cc:3037] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:08.115908 13789 raft_consensus.cc:504] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:08.116021 13789 leader_election.cc:288] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [CANDIDATE]: Term 1 election: Requested vote from peers a5beac2c783a462f86f6fd656a3cfba4 (127.11.220.193:40007), cc765a2a9f514edba4c54805c53b41a6 (127.11.220.195:33947)
I20260925 02:27:08.116187 13712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ecf291895344d5bc66930a40a66793" candidate_uuid: "f2c6fa29942941eea69bb6c9487a633a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc765a2a9f514edba4c54805c53b41a6"
I20260925 02:27:08.116254 13712 raft_consensus.cc:3037] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:08.116200 13429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90ecf291895344d5bc66930a40a66793" candidate_uuid: "f2c6fa29942941eea69bb6c9487a633a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5beac2c783a462f86f6fd656a3cfba4"
I20260925 02:27:08.116281 13429 raft_consensus.cc:3037] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:08.116892 13712 raft_consensus.cc:2454] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c6fa29942941eea69bb6c9487a633a in term 1.
I20260925 02:27:08.116995 13429 raft_consensus.cc:2454] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2c6fa29942941eea69bb6c9487a633a in term 1.
I20260925 02:27:08.117086 13502 leader_election.cc:302] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc765a2a9f514edba4c54805c53b41a6, f2c6fa29942941eea69bb6c9487a633a; no voters:
I20260925 02:27:08.117201 13789 raft_consensus.cc:2781] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:08.117261 13789 raft_consensus.cc:686] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [term 1 LEADER]: Becoming Leader. State: Replica: f2c6fa29942941eea69bb6c9487a633a, State: Running, Role: LEADER
I20260925 02:27:08.117305 13789 consensus_queue.cc:237] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } }
I20260925 02:27:08.117563 13787 tablet_replica.cc:439] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: TxnStatusTablet state changed. Reason: New leader f2c6fa29942941eea69bb6c9487a633a. Latest consensus state: current_term: 1 leader_uuid: "f2c6fa29942941eea69bb6c9487a633a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } } }
I20260925 02:27:08.117630 13787 tablet_replica.cc:442] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:27:08.117794 13821 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 02:27:08.117910 13821 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 02:27:08.117882 13271 catalog_manager.cc:5495] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a reported cstate change: term changed from 0 to 1, leader changed from <none> to f2c6fa29942941eea69bb6c9487a633a (127.11.220.194). New cstate: current_term: 1 leader_uuid: "f2c6fa29942941eea69bb6c9487a633a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } health_report { overall_health: UNKNOWN } } }
I20260925 02:27:08.128749 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:08.126496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44920 (local address 127.11.220.194:34781)
0925 02:27:08.126551 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:08.126553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:08.126563 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:27:08.126651 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:08.126652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:08.126662 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:08.126715 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:08.126835 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.127294 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.128034 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.128235 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.128306 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:08.128494 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:08.128536 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:08.128596 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:08.128663 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 02:27:08.128685 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 02:27:08.128777 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:08.126442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:34781 (local address 127.0.0.1:44920)
0925 02:27:08.126524 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:08.126536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:08.126582 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:08.126726 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:08.126726 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:08.126727 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:08.126775 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:08.126777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.127375 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.127376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:08.127948 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:08.127949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.128306 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.128307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:08.128380 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:08.128395 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:08.128597 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:08.128607 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 02:27:08.128658 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:27:08.128706 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 02:27:08.129601 13712 raft_consensus.cc:1261] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6 [term 1 FOLLOWER]: Refusing update from remote peer f2c6fa29942941eea69bb6c9487a633a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:08.129753 13429 raft_consensus.cc:1261] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4 [term 1 FOLLOWER]: Refusing update from remote peer f2c6fa29942941eea69bb6c9487a633a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:08.129833 13789 consensus_queue.cc:1035] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:08.129916 13787 consensus_queue.cc:1035] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:08.130366 13785 tablet_replica.cc:439] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: TxnStatusTablet state changed. Reason: New leader f2c6fa29942941eea69bb6c9487a633a. Latest consensus state: current_term: 1 leader_uuid: "f2c6fa29942941eea69bb6c9487a633a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } } }
I20260925 02:27:08.130468 13785 tablet_replica.cc:442] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:08.130631 13786 tablet_replica.cc:439] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: TxnStatusTablet state changed. Reason: New leader f2c6fa29942941eea69bb6c9487a633a. Latest consensus state: current_term: 1 leader_uuid: "f2c6fa29942941eea69bb6c9487a633a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } } }
I20260925 02:27:08.130755 13786 tablet_replica.cc:442] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:08.131084 13789 tablet_replica.cc:439] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f2c6fa29942941eea69bb6c9487a633a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } } }
I20260925 02:27:08.131178 13789 tablet_replica.cc:442] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:27:08.131227 13785 tablet_replica.cc:439] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f2c6fa29942941eea69bb6c9487a633a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } } }
I20260925 02:27:08.131304 13785 tablet_replica.cc:442] T 90ecf291895344d5bc66930a40a66793 P a5beac2c783a462f86f6fd656a3cfba4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:08.132290 13789 tablet_replica.cc:439] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f2c6fa29942941eea69bb6c9487a633a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } } }
I20260925 02:27:08.132382 13789 tablet_replica.cc:442] T 90ecf291895344d5bc66930a40a66793 P f2c6fa29942941eea69bb6c9487a633a: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:27:08.132838 13786 tablet_replica.cc:439] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f2c6fa29942941eea69bb6c9487a633a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5beac2c783a462f86f6fd656a3cfba4" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40007 } } peers { permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 } } peers { permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 } } }
I20260925 02:27:08.132920 13786 tablet_replica.cc:442] T 90ecf291895344d5bc66930a40a66793 P cc765a2a9f514edba4c54805c53b41a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:08.135804 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:08.133137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.11.220.193:40007)
0925 02:27:08.133179 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:08.133198 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:08.133206 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:08.133256 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:08.133257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:08.133276 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:08.133339 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:08.133540 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.134049 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.135107 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.135280 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.135330 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:08.135505 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:08.135597 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:08.135650 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:08.135723 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 02:27:08.135738 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:27:08.140409 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:08.138156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:38635 (local address 127.11.220.194:34781)
0925 02:27:08.138193 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:08.138195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:08.138302 (+ 107us) server_negotiation.cc:407] Connection header received
0925 02:27:08.138365 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:08.138366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:08.138375 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:08.138434 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:08.138579 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.138994 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.139712 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.139882 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.139946 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:08.140170 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:08.140199 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:08.140266 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:08.140333 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 02:27:08.140351 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:08.140516 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:08.138046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:34781 (local address 127.11.220.193:38635)
0925 02:27:08.138230 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:08.138239 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:08.138316 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:08.138450 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:08.138450 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:08.138452 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:08.138504 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:08.138505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.139069 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.139070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:08.139634 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:08.139636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.139976 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.139977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:08.140048 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:08.140060 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:08.140288 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:08.140290 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:08.140332 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 02:27:08.140370 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260925 02:27:08.142470 13712 raft_consensus.cc:1261] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6 [term 1 FOLLOWER]: Refusing update from remote peer a5beac2c783a462f86f6fd656a3cfba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:08.142711 13809 consensus_queue.cc:1035] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc765a2a9f514edba4c54805c53b41a6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:08.142776 13570 raft_consensus.cc:1261] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a [term 1 FOLLOWER]: Refusing update from remote peer a5beac2c783a462f86f6fd656a3cfba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:08.143168 13809 consensus_queue.cc:1035] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2c6fa29942941eea69bb6c9487a633a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 34781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:08.151103 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:08.148097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:40007 (local address 127.11.220.194:36393)
0925 02:27:08.148250 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:08.148259 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:08.148328 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:08.148440 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:08.148441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:08.148464 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:08.148500 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:08.148501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.149523 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.149525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:08.150255 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:08.150257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.150773 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.150774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:08.150818 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:08.150829 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:08.150994 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:08.150995 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 02:27:08.151025 (+ 30us) client_negotiation.cc:240] Negotiation successful
0925 02:27:08.151053 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
I20260925 02:27:08.151105 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:08.148147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:36393 (local address 127.11.220.193:40007)
0925 02:27:08.148179 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:08.148181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:08.148319 (+ 138us) server_negotiation.cc:407] Connection header received
0925 02:27:08.148382 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:08.148383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:08.148391 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:08.148429 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:08.148558 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.149220 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.150370 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.150664 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.150721 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:08.150901 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:08.150934 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:08.150996 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:08.151027 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 02:27:08.151045 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 02:27:08.157701 13271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:08.157802 13271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:08.158437 13271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b326b6cc470489a91ca71a8e55e60d6: Sending DeleteTablet for 3 replicas of tablet e6d64ebcb1da4ec99a4e270ecf24c5ab
I20260925 02:27:08.158650 13409 tablet_service.cc:1514] Processing DeleteTablet for tablet e6d64ebcb1da4ec99a4e270ecf24c5ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:08 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:42868
I20260925 02:27:08.158726 13801 tablet_replica.cc:330] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: stopping tablet replica
I20260925 02:27:08.158715 13692 tablet_service.cc:1514] Processing DeleteTablet for tablet e6d64ebcb1da4ec99a4e270ecf24c5ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:08 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:56084
I20260925 02:27:08.158766 13801 raft_consensus.cc:2229] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:08.158799 13803 tablet_replica.cc:330] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: stopping tablet replica
I20260925 02:27:08.158834 13801 raft_consensus.cc:2258] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:08.158849 13803 raft_consensus.cc:2229] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:08.158841 13550 tablet_service.cc:1514] Processing DeleteTablet for tablet e6d64ebcb1da4ec99a4e270ecf24c5ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:08 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:44910
I20260925 02:27:08.158881 13803 raft_consensus.cc:2258] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:08.158946 13802 tablet_replica.cc:330] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a: stopping tablet replica
I20260925 02:27:08.158991 13802 raft_consensus.cc:2229] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:08.159019 13802 raft_consensus.cc:2258] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:08.159129 13801 ts_tablet_manager.cc:1905] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:08.159370 13803 ts_tablet_manager.cc:1905] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:08.160673 13801 ts_tablet_manager.cc:1918] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:08.160676 13802 ts_tablet_manager.cc:1905] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:08.160742 13801 log.cc:1196] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/e6d64ebcb1da4ec99a4e270ecf24c5ab
I20260925 02:27:08.160913 13803 ts_tablet_manager.cc:1918] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:08.160960 13803 log.cc:1196] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/e6d64ebcb1da4ec99a4e270ecf24c5ab
I20260925 02:27:08.160956 13801 ts_tablet_manager.cc:1939] T e6d64ebcb1da4ec99a4e270ecf24c5ab P a5beac2c783a462f86f6fd656a3cfba4: Deleting consensus metadata
I20260925 02:27:08.161146 13803 ts_tablet_manager.cc:1939] T e6d64ebcb1da4ec99a4e270ecf24c5ab P cc765a2a9f514edba4c54805c53b41a6: Deleting consensus metadata
I20260925 02:27:08.161253 13256 catalog_manager.cc:4841] TS a5beac2c783a462f86f6fd656a3cfba4 (127.11.220.193:40007): tablet e6d64ebcb1da4ec99a4e270ecf24c5ab (table test-table [id=29710a174de44a35aadc7b1955b77daf]) successfully deleted
I20260925 02:27:08.161422 13256 catalog_manager.cc:4841] TS cc765a2a9f514edba4c54805c53b41a6 (127.11.220.195:33947): tablet e6d64ebcb1da4ec99a4e270ecf24c5ab (table test-table [id=29710a174de44a35aadc7b1955b77daf]) successfully deleted
I20260925 02:27:08.162099 13802 ts_tablet_manager.cc:1918] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:08.162150 13802 log.cc:1196] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/e6d64ebcb1da4ec99a4e270ecf24c5ab
I20260925 02:27:08.162348 13802 ts_tablet_manager.cc:1939] T e6d64ebcb1da4ec99a4e270ecf24c5ab P f2c6fa29942941eea69bb6c9487a633a: Deleting consensus metadata
I20260925 02:27:08.162797 13256 catalog_manager.cc:4841] TS f2c6fa29942941eea69bb6c9487a633a (127.11.220.194:34781): tablet e6d64ebcb1da4ec99a4e270ecf24c5ab (table test-table [id=29710a174de44a35aadc7b1955b77daf]) successfully deleted
Sep 25 02:27:08 dist-test-slave-dn4n krb5kdc[13226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303227, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:27:08.174476 13774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:08.164666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42886 (local address 127.11.220.193:40007)
0925 02:27:08.164700 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:08.164702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:08.164966 (+ 264us) server_negotiation.cc:407] Connection header received
0925 02:27:08.165020 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:08.165021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:08.165031 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:08.165087 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:08.165765 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.166444 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.167503 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.167785 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.167825 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:08.171162 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:08.171176 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:08.171176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:08.171182 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:08.173037 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:08.173524 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:08.173527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:08.173527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:08.173615 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:08.173869 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:08.173870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:08.173871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:08.174048 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:08.174196 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:08.174263 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 02:27:08.174361 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 02:27:08.174791 13470 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42886
I20260925 02:27:08.182955 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:08.175778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.11.220.254:40359)
0925 02:27:08.175819 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:08.175820 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:08.175943 (+ 123us) server_negotiation.cc:407] Connection header received
0925 02:27:08.175989 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:08.175990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:08.175999 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:08.176067 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:08.176632 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.177308 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.178193 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:08.178483 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:08.178551 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:08.179837 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:08.179838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:08.179838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:08.179844 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:08.181766 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:08.182145 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:08.182146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:08.182146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:08.182214 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:08.182464 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:08.182465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:08.182465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:08.182592 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:08.182692 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:08.182798 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 02:27:08.182869 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 02:27:08.183302 13271 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54782
I20260925 02:27:08.183944 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 13344
I20260925 02:27:08.191308 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 13485
I20260925 02:27:08.197000 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 13626
I20260925 02:27:08.202373 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 13239
2026-09-25T02:27:08Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2181 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:08 dist-test-slave-dn4n krb5kdc[13843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:08 dist-test-slave-dn4n krb5kdc[13843](info): set up 2 sockets
Sep 25 02:27:08 dist-test-slave-dn4n krb5kdc[13843](info): commencing operation
krb5kdc: starting...
W20260925 02:27:09.172272 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:09.272907 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:46803
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:46803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:09.344055 13856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:09.344218 13856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:09.344235 13856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:09.345293 13856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:09.345326 13856 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 02:27:09.345340 13856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:09.345351 13856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:09.345376 13856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:09.345386 13856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:09.346624 13856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:46803
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:46803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:09.346791 13856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:09.346928 13856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:09.351091 13864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:09.351253 13856 server_base.cc:1034] running on GCE node
W20260925 02:27:09.351169 13862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:09.351111 13861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:09.351580 13856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:09.351840 13856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:09.352973 13856 hybrid_clock.cc:648] HybridClock initialized: now 1790303229352960 us; error 27 us; skew 500 ppm
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:09.355177 13856 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:09.355614 13856 webserver.cc:460] Webserver started at http://127.11.220.254:42331/ using document root <none> and password file <none>
I20260925 02:27:09.355818 13856 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:09.355856 13856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:09.355993 13856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:09.356948 13856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "6e9e64fb415d474fa97f5149ce22686d"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "3bca540c44d48ac0a1b91a3326c12ddc"
server_key_iv: "074ae61307c6767c56176665f9fd17b1"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.357260 13856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "6e9e64fb415d474fa97f5149ce22686d"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "3bca540c44d48ac0a1b91a3326c12ddc"
server_key_iv: "074ae61307c6767c56176665f9fd17b1"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.358389 13856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:09.358985 13871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:09.359149 13856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:09.359237 13856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "6e9e64fb415d474fa97f5149ce22686d"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "3bca540c44d48ac0a1b91a3326c12ddc"
server_key_iv: "074ae61307c6767c56176665f9fd17b1"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.359308 13856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:09.375789 13856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:09.377239 13856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:09.377372 13856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:09.381389 13856 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:46803
I20260925 02:27:09.381440 13933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:46803 every 8 connection(s)
I20260925 02:27:09.381703 13856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:09.382205 13934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:09.384159 13934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d: Bootstrap starting.
I20260925 02:27:09.384688 13934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:09.384943 13934 log.cc:824] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:09.385514 13934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d: No bootstrap required, opened a new log
I20260925 02:27:09.386636 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 13856
I20260925 02:27:09.386721 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:09.386881 12147 external_mini_cluster.cc:1428] Setting key 11e07e266efea0ea8b9330190ceb07f6
I20260925 02:27:09.387750 13934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9e64fb415d474fa97f5149ce22686d" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 46803 } }
I20260925 02:27:09.387881 13934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:09.387928 13934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e9e64fb415d474fa97f5149ce22686d, State: Initialized, Role: FOLLOWER
I20260925 02:27:09.388084 13934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9e64fb415d474fa97f5149ce22686d" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 46803 } }
I20260925 02:27:09.388163 13934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:09.388201 13934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:09.388264 13934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:09.388810 13934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9e64fb415d474fa97f5149ce22686d" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 46803 } }
I20260925 02:27:09.388916 13934 leader_election.cc:302] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e9e64fb415d474fa97f5149ce22686d; no voters:
I20260925 02:27:09.389050 13934 leader_election.cc:288] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:09.389078 13939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:09.389170 13939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [term 1 LEADER]: Becoming Leader. State: Replica: 6e9e64fb415d474fa97f5149ce22686d, State: Running, Role: LEADER
I20260925 02:27:09.389240 13939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9e64fb415d474fa97f5149ce22686d" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 46803 } }
I20260925 02:27:09.389278 13934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:09.389525 13940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e9e64fb415d474fa97f5149ce22686d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9e64fb415d474fa97f5149ce22686d" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 46803 } } }
I20260925 02:27:09.389760 13940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:09.389708 13941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e9e64fb415d474fa97f5149ce22686d. Latest consensus state: current_term: 1 leader_uuid: "6e9e64fb415d474fa97f5149ce22686d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9e64fb415d474fa97f5149ce22686d" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 46803 } } }
I20260925 02:27:09.390057 13941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:09.390260 13948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:09.390750 13948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:09.392313 13948 catalog_manager.cc:1347] Generated new cluster ID: 7764d33a223c4da2860fac467b0077f4
I20260925 02:27:09.392381 13948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:09.397334 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.387332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57450 (local address 127.11.220.254:46803)
0925 02:27:09.387519 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.387526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.387548 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:27:09.387739 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.387745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.387783 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.387910 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:09.388658 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.389806 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.390943 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.391482 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.391589 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.394135 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:09.394139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:09.394139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:09.394150 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:09.395878 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.396224 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.396228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.396228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.396351 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.396552 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.396553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.396553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.396798 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:09.396916 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.397056 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.397170 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260925 02:27:09.414683 13948 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:09.415262 13948 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:09.423381 13948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d: Generated new TSK 0
I20260925 02:27:09.423544 13948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:09.445402 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:46803
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:09.516800 13960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:09.517011 13960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:09.517052 13960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:09.518141 13960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:09.518198 13960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:09.518230 13960 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 02:27:09.518273 13960 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:27:09.518301 13960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:09.519629 13960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.220.254:46803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:09.519855 13960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:09.520073 13960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:09.524556 13967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:09.524573 13966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:09.524590 13960 server_base.cc:1034] running on GCE node
W20260925 02:27:09.524564 13969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:09.524924 13960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:09.525158 13960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:09.526315 13960 hybrid_clock.cc:648] HybridClock initialized: now 1790303229526306 us; error 35 us; skew 500 ppm
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:09.528383 13960 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:09.528764 13960 webserver.cc:460] Webserver started at http://127.11.220.193:44043/ using document root <none> and password file <none>
I20260925 02:27:09.528975 13960 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:09.529034 13960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:09.529155 13960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:09.530077 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "b3e97d413eb8491db9b232970ec8b95d"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "60ed9f3882cbe00d89d1994414a91f96"
server_key_iv: "30ae44560d2508531d1b0d87baba68f5"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.530392 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "b3e97d413eb8491db9b232970ec8b95d"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "60ed9f3882cbe00d89d1994414a91f96"
server_key_iv: "30ae44560d2508531d1b0d87baba68f5"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.531539 13960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:09.532212 13976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:09.532375 13960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:09.532477 13960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "b3e97d413eb8491db9b232970ec8b95d"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "60ed9f3882cbe00d89d1994414a91f96"
server_key_iv: "30ae44560d2508531d1b0d87baba68f5"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.532559 13960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:09.546553 13960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:09.548544 13960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:09.548674 13960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:09.549118 13960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:09.549160 13960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:09.549185 13960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:09.549209 13960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:09.555284 13960 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:35299
I20260925 02:27:09.555341 14093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:35299 every 8 connection(s)
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:09.555605 13960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:09.558378 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.549488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:40071 (local address 127.11.220.254:46803)
0925 02:27:09.549525 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.549527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.549700 (+ 173us) server_negotiation.cc:407] Connection header received
0925 02:27:09.550084 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.550085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.550101 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.550181 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:09.550854 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.551612 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.553085 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.553417 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.553485 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.555947 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:09.555949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:09.555949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:09.555956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:09.557343 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.557686 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.557687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.557687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.557735 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.557930 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.557931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.557931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.558021 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:09.558069 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.558262 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.558318 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:09.558571 13985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.549464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:46803 (local address 127.11.220.193:40071)
0925 02:27:09.549607 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.549636 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.549990 (+ 354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.550220 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.550222 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.550484 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:09.550758 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.550763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.551729 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.551730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.552984 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.552987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.553522 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.553523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.553651 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.554162 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:09.554176 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:09.555865 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:09.557410 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.557414 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.557416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.557618 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.557789 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.557790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.557790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.557879 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.558073 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:09.558077 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:09.558175 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.558250 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.558353 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260925 02:27:09.559047 14094 heartbeater.cc:347] Connected to a master server at 127.11.220.254:46803
I20260925 02:27:09.559122 14094 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:09.559310 14094 heartbeater.cc:511] Master 127.11.220.254:46803 requested a full tablet report, sending...
I20260925 02:27:09.559332 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 13960
I20260925 02:27:09.559433 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:09.559640 12147 external_mini_cluster.cc:1428] Setting key 4ac7b512a8e1ca27a3fbb36e3e8335bc
I20260925 02:27:09.559785 13888 ts_manager.cc:194] Registered new tserver with Master: b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299)
I20260925 02:27:09.560727 13888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:40071
I20260925 02:27:09.566309 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.559945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:54603 (local address 127.11.220.254:46803)
0925 02:27:09.560280 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.560282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.560287 (+ 5us) server_negotiation.cc:407] Connection header received
0925 02:27:09.560299 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.560300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.560306 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.560365 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:09.560694 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.561337 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.561998 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.562201 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.562266 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.563125 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:09.563126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:09.563127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:09.563134 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:09.565303 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.565626 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.565627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.565627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.565674 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.565861 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.565862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.565862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.565959 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:09.566021 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.566168 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.566230 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322}
I20260925 02:27:09.566322 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.559956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:46803 (local address 127.11.220.193:54603)
0925 02:27:09.560076 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.560086 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.560131 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.560382 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.560383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.560570 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:09.560619 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.560621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.561449 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.561450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.561884 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.561886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.562275 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.562276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.562339 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.562741 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:09.562745 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:09.563010 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:09.565395 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.565396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.565397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.565553 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.565743 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.565744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.565744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.565796 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.566025 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:09.566026 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:09.566062 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.566145 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.566232 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:09.580883 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:46803
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:09.652403 14103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:09.652609 14103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:09.652645 14103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:09.653692 14103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:09.653748 14103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:09.653781 14103 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 02:27:09.653822 14103 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:27:09.653858 14103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:09.655167 14103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.220.254:46803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:09.655390 14103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:09.655583 14103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:09.659935 14111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:09.659950 14108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:09.659925 14109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:09.660110 14103 server_base.cc:1034] running on GCE node
I20260925 02:27:09.660302 14103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:09.660568 14103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:09.661729 14103 hybrid_clock.cc:648] HybridClock initialized: now 1790303229661720 us; error 35 us; skew 500 ppm
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:09.663947 14103 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:09.664356 14103 webserver.cc:460] Webserver started at http://127.11.220.194:34579/ using document root <none> and password file <none>
I20260925 02:27:09.664594 14103 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:09.664657 14103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:09.664777 14103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:09.665683 14103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "caf0a422dec44242bb926b891ef4d0df"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "c675b710073b9d8f007845886aec185a"
server_key_iv: "972505c4bacb087155b8696d66d53655"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.666008 14103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "caf0a422dec44242bb926b891ef4d0df"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "c675b710073b9d8f007845886aec185a"
server_key_iv: "972505c4bacb087155b8696d66d53655"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.667191 14103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:09.667829 14118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:09.667987 14103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:09.668077 14103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "caf0a422dec44242bb926b891ef4d0df"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "c675b710073b9d8f007845886aec185a"
server_key_iv: "972505c4bacb087155b8696d66d53655"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.668177 14103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:09.687899 14103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:09.690119 14103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:09.690280 14103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:09.690726 14103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:09.690781 14103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:09.690819 14103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:09.690842 14103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:09.697324 14103 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:42007
I20260925 02:27:09.697388 14234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:42007 every 8 connection(s)
I20260925 02:27:09.697638 14103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:09.700232 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.691211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:58799 (local address 127.11.220.254:46803)
0925 02:27:09.691262 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.691265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.691478 (+ 213us) server_negotiation.cc:407] Connection header received
0925 02:27:09.691891 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.691892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.691905 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.691982 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:09.692788 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.693629 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.694995 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.695279 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.695361 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.697761 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:09.697763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:09.697763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:09.697770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:09.698951 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.699319 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.699320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.699320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.699380 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.699606 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.699607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.699607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.699718 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:09.699784 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.700035 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.700135 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:27:09.700382 14127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.691193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:46803 (local address 127.11.220.194:58799)
0925 02:27:09.691359 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.691395 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.691804 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.692047 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.692050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.692374 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:09.692656 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.692663 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.693736 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.693737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.694892 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.694895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.695384 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.695385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.695533 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.696015 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:09.696023 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:09.697656 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:09.699029 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.699036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.699038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.699238 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.699439 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.699440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.699440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.699539 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.699776 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:09.699783 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:09.699905 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.700004 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.700165 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":98,"threads_started":1}
I20260925 02:27:09.700839 14235 heartbeater.cc:347] Connected to a master server at 127.11.220.254:46803
I20260925 02:27:09.700937 14235 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:09.701104 14235 heartbeater.cc:511] Master 127.11.220.254:46803 requested a full tablet report, sending...
I20260925 02:27:09.701481 13888 ts_manager.cc:194] Registered new tserver with Master: caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007)
I20260925 02:27:09.701918 13888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:58799
I20260925 02:27:09.704910 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 14103
I20260925 02:27:09.704998 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:09.705205 12147 external_mini_cluster.cc:1428] Setting key ec5f9d3a2d11b7a52a526fa240c63270
I20260925 02:27:09.706676 14241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.701756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:46803 (local address 127.11.220.194:42791)
0925 02:27:09.701900 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.701911 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.701980 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.702131 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.702132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.702313 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:09.702372 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.702374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.703006 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.703006 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.703438 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.703439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.703703 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.703704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.703751 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.704131 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:09.704136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:09.704368 (+ 232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:09.705837 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.705838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.705838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.705980 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.706154 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.706154 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.706154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.706199 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.706480 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:09.706481 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:09.706511 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.706554 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.706617 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
I20260925 02:27:09.706756 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.701812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:42791 (local address 127.11.220.254:46803)
0925 02:27:09.701850 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.701852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.701991 (+ 139us) server_negotiation.cc:407] Connection header received
0925 02:27:09.702066 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.702067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.702077 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.702125 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:09.702462 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.702933 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.703498 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.703648 (+ 150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.703692 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.704427 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:09.704428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:09.704428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:09.704432 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:09.705765 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.706043 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.706044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.706044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.706094 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.706249 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.706250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.706250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.706425 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:09.706485 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.706575 (+ 90us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.706642 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:09.726423 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:46803
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:09.801819 14244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:09.802026 14244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:09.802064 14244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:09.803136 14244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:09.803191 14244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:09.803225 14244 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 02:27:09.803267 14244 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:27:09.803301 14244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:09.804678 14244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.220.254:46803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:09.804898 14244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:09.805106 14244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:09.809510 14250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:09.809525 14244 server_base.cc:1034] running on GCE node
W20260925 02:27:09.809512 14252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:09.809530 14249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:09.809898 14244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:09.810143 14244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:09.811302 14244 hybrid_clock.cc:648] HybridClock initialized: now 1790303229811283 us; error 30 us; skew 500 ppm
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:09.813527 14244 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:09.813880 14244 webserver.cc:460] Webserver started at http://127.11.220.195:35257/ using document root <none> and password file <none>
I20260925 02:27:09.814087 14244 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:09.814147 14244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:09.814285 14244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:09.815192 14244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "01156eb3e7c3401c8f7c35234365615c"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "3c1643e772e505cc98358d5a451db0a2"
server_key_iv: "efe72e175e7ebf4e19ddd5749f0414cf"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.815555 14244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "01156eb3e7c3401c8f7c35234365615c"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "3c1643e772e505cc98358d5a451db0a2"
server_key_iv: "efe72e175e7ebf4e19ddd5749f0414cf"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.816862 14244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:27:09.817559 14259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:09.817723 14244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:09.817816 14244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "01156eb3e7c3401c8f7c35234365615c"
format_stamp: "Formatted at 2026-09-25 02:27:09 on dist-test-slave-dn4n"
server_key: "3c1643e772e505cc98358d5a451db0a2"
server_key_iv: "efe72e175e7ebf4e19ddd5749f0414cf"
server_key_version: "encryptionkey@0"
I20260925 02:27:09.817914 14244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:09.853329 14244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:09.855445 14244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:09.855618 14244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:09.856310 14244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:09.856356 14244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:09.856389 14244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:09.856412 14244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:09.863400 14244 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:33231
I20260925 02:27:09.863476 14376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:33231 every 8 connection(s)
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:09.863744 14244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:09.867317 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.856677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:53999 (local address 127.11.220.254:46803)
0925 02:27:09.856726 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.856729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.856979 (+ 250us) server_negotiation.cc:407] Connection header received
0925 02:27:09.857703 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.857705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.857719 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.857755 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:09.858723 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.859490 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.861162 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.861460 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.861502 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.864230 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:09.864231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:09.864231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:09.864239 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:09.865889 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.866309 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.866311 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.866311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.866378 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.866630 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.866632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.866632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.866773 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:09.866840 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.867113 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.867205 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 02:27:09.867419 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.856671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:46803 (local address 127.11.220.195:53999)
0925 02:27:09.856832 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.856880 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.857566 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.857945 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.857949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.858319 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:09.858597 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.858603 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.859625 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.859626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.861062 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.861066 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.861594 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.861595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.861802 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.862384 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:09.862393 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:09.864159 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:09.865963 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.865968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.865970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.866235 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.866454 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.866455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.866455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.866564 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.866835 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:09.866841 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:09.866978 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.867069 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.867201 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260925 02:27:09.868072 14377 heartbeater.cc:347] Connected to a master server at 127.11.220.254:46803
I20260925 02:27:09.868161 14377 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:09.868359 14377 heartbeater.cc:511] Master 127.11.220.254:46803 requested a full tablet report, sending...
I20260925 02:27:09.868753 13888 ts_manager.cc:194] Registered new tserver with Master: 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231)
I20260925 02:27:09.869285 13888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:53999
I20260925 02:27:09.870507 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 14244
I20260925 02:27:09.870622 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:09.870826 12147 external_mini_cluster.cc:1428] Setting key 163c69cd58cf2fe6b21fa7706f379a88
I20260925 02:27:09.871640 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:27:09.874627 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.869134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:46803 (local address 127.11.220.195:41203)
0925 02:27:09.869296 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.869304 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.869359 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.869497 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.869498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.869708 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:09.869761 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.869763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.870348 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.870349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.871064 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.871066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.871425 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.871425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.871488 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.871862 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:09.871867 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:09.872173 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:09.873808 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.873809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.873809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.873957 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.874140 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:09.874141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:09.874141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:09.874186 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:09.874422 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:09.874422 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:09.874452 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.874497 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.874559 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":121,"threads_started":1}
I20260925 02:27:09.874635 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.869177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:41203 (local address 127.11.220.254:46803)
0925 02:27:09.869208 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.869210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.869347 (+ 137us) server_negotiation.cc:407] Connection header received
0925 02:27:09.869429 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.869430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.869440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.869490 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:09.869820 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.870279 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.871124 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.871371 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.871425 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.872258 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:09.872259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:09.872260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:09.872266 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:09.873732 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.874022 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.874023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.874024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.874081 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.874236 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.874237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.874237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.874362 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:09.874428 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.874520 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.874584 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 02:27:09 dist-test-slave-dn4n krb5kdc[13843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:09.892917 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.886683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57454 (local address 127.11.220.254:46803)
0925 02:27:09.886729 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.886730 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.886759 (+ 29us) server_negotiation.cc:407] Connection header received
0925 02:27:09.886777 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.886778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.886797 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.886832 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:09.887335 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.887774 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.888419 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.888623 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.888674 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.890567 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:09.890568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:09.890569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:09.890574 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:09.891840 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.892159 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.892160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.892160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.892213 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:09.892398 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:09.892399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:09.892399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:09.892550 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:09.892639 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.892766 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.892830 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 02:27:09.894160 13888 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:09.895601 13888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:09.901297 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.897491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:33231 (local address 127.0.0.1:34458)
0925 02:27:09.897945 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.897953 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.898013 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.898242 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.898244 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.898246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:09.898301 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.898302 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.899044 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.899045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.899983 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.899985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.901019 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.901020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.901107 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.901119 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.901188 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.901232 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":55,"threads_started":1}
I20260925 02:27:09.901333 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.897768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.11.220.195:33231)
0925 02:27:09.897893 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.897897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.898012 (+ 115us) server_negotiation.cc:407] Connection header received
0925 02:27:09.898084 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.898086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.898100 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.898179 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:09.898366 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.898970 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.900215 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.900940 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.901011 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.901060 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.901210 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.901244 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
I20260925 02:27:09.901690 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.897683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:42007 (local address 127.0.0.1:59222)
0925 02:27:09.898105 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.898112 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.898167 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.898409 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.898409 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.898411 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:09.898441 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.898442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.899530 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.899531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.900320 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.900322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.901427 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.901428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.901493 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.901497 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.901548 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.901584 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":19,"thread_start_us":42,"threads_started":1}
I20260925 02:27:09.901705 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.897789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.11.220.194:42007)
0925 02:27:09.897920 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.897923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.898224 (+ 301us) server_negotiation.cc:407] Connection header received
0925 02:27:09.898264 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.898266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.898279 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.898343 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:09.898500 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.899451 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.900402 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.901126 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.901219 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.901269 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.901579 (+ 310us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.901618 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260925 02:27:09.901963 14310 tablet_service.cc:1467] Processing CreateTablet for tablet b02320a3f86b430f84862acc1a0ec9c7 (DEFAULT_TABLE table=test-table [id=724e73da332147cb94cff4e10d1d43b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:09.902323 14310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b02320a3f86b430f84862acc1a0ec9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:09.902470 14169 tablet_service.cc:1467] Processing CreateTablet for tablet b02320a3f86b430f84862acc1a0ec9c7 (DEFAULT_TABLE table=test-table [id=724e73da332147cb94cff4e10d1d43b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:09.902776 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b02320a3f86b430f84862acc1a0ec9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:09.904914 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.897592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.11.220.193:35299)
0925 02:27:09.897823 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.897828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.898011 (+ 183us) server_negotiation.cc:407] Connection header received
0925 02:27:09.898093 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.898095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.898107 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.898177 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:09.898445 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.899423 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.900704 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.901320 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.904507 (+ 3187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.904559 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.904803 (+ 244us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.904842 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":80,"threads_started":1}
I20260925 02:27:09.905097 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.897490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:35299 (local address 127.0.0.1:52646)
0925 02:27:09.897914 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.897946 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.898021 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.898243 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.898249 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.898252 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:09.898361 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.898363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.899509 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.899510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.900614 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.900617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.904651 (+ 4034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.904652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.904716 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.904719 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.904991 (+ 272us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.905034 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":2,"thread_start_us":50,"threads_started":1}
I20260925 02:27:09.905403 14397 tablet_bootstrap.cc:492] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: Bootstrap starting.
I20260925 02:27:09.905920 14398 tablet_bootstrap.cc:492] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: Bootstrap starting.
I20260925 02:27:09.906194 14397 tablet_bootstrap.cc:654] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:09.906438 14398 tablet_bootstrap.cc:654] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:09.906471 14397 log.cc:824] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:09.906527 14027 tablet_service.cc:1467] Processing CreateTablet for tablet b02320a3f86b430f84862acc1a0ec9c7 (DEFAULT_TABLE table=test-table [id=724e73da332147cb94cff4e10d1d43b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:09.906678 14398 log.cc:824] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:09.906788 14027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b02320a3f86b430f84862acc1a0ec9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:09.907209 14397 tablet_bootstrap.cc:492] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: No bootstrap required, opened a new log
I20260925 02:27:09.907259 14397 ts_tablet_manager.cc:1397] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:09.907289 14398 tablet_bootstrap.cc:492] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: No bootstrap required, opened a new log
I20260925 02:27:09.907358 14398 ts_tablet_manager.cc:1397] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:09.908792 14398 raft_consensus.cc:348] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } }
I20260925 02:27:09.908939 14398 raft_consensus.cc:374] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:09.908977 14398 raft_consensus.cc:729] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf0a422dec44242bb926b891ef4d0df, State: Initialized, Role: FOLLOWER
I20260925 02:27:09.909091 14398 consensus_queue.cc:260] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } }
I20260925 02:27:09.909336 14235 heartbeater.cc:503] Master 127.11.220.254:46803 was elected leader, sending a full tablet report...
I20260925 02:27:09.909260 14397 raft_consensus.cc:348] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } }
I20260925 02:27:09.909420 14397 raft_consensus.cc:374] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:09.909463 14397 raft_consensus.cc:729] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01156eb3e7c3401c8f7c35234365615c, State: Initialized, Role: FOLLOWER
I20260925 02:27:09.909564 14397 consensus_queue.cc:260] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } }
I20260925 02:27:09.909332 14398 ts_tablet_manager.cc:1428] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:27:09.910058 14397 ts_tablet_manager.cc:1428] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:27:09.910128 14377 heartbeater.cc:503] Master 127.11.220.254:46803 was elected leader, sending a full tablet report...
I20260925 02:27:09.910267 14402 tablet_bootstrap.cc:492] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: Bootstrap starting.
I20260925 02:27:09.911005 14402 tablet_bootstrap.cc:654] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:09.911273 14402 log.cc:824] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:09.911847 14402 tablet_bootstrap.cc:492] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: No bootstrap required, opened a new log
I20260925 02:27:09.911907 14402 ts_tablet_manager.cc:1397] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:09.913208 14402 raft_consensus.cc:348] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } }
I20260925 02:27:09.913309 14402 raft_consensus.cc:374] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:09.913327 14402 raft_consensus.cc:729] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3e97d413eb8491db9b232970ec8b95d, State: Initialized, Role: FOLLOWER
I20260925 02:27:09.913455 14402 consensus_queue.cc:260] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } }
I20260925 02:27:09.913666 14402 ts_tablet_manager.cc:1428] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:27:09.913724 14094 heartbeater.cc:503] Master 127.11.220.254:46803 was elected leader, sending a full tablet report...
W20260925 02:27:09.948175 14236 tablet.cc:2369] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:09.969321 14403 raft_consensus.cc:482] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:09.969542 14403 raft_consensus.cc:504] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } }
I20260925 02:27:09.969853 14403 leader_election.cc:288] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299), caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007)
I20260925 02:27:09.972728 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.970129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:35705 (local address 127.11.220.193:35299)
0925 02:27:09.970171 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.970173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.970182 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:09.970255 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.970256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.970264 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.970339 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:09.970488 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.970937 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.972022 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.972365 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.972443 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.972452 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.972625 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.972659 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:27:09.972728 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.970020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:35299 (local address 127.11.220.195:35705)
0925 02:27:09.970089 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.970101 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.970160 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.970344 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.970345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.970347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:09.970415 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.970417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.971013 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.971014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.971939 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.971942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.972470 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.972471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.972554 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.972557 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.972620 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.972659 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 02:27:09.972818 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.970101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:51795 (local address 127.11.220.194:42007)
0925 02:27:09.970138 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:09.970140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:09.970319 (+ 179us) server_negotiation.cc:407] Connection header received
0925 02:27:09.970383 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:09.970384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:09.970391 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:09.970433 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:09.970595 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.971004 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.972073 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.972429 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.972538 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.972546 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:09.972716 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 02:27:09.972755 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:27:09.972837 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:09.970020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:42007 (local address 127.11.220.195:51795)
0925 02:27:09.970255 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:09.970270 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:09.970327 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:09.970433 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:09.970434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:09.970435 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:09.970528 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.970529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.971080 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.971081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.971955 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:09.971958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:09.972543 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:09.972544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:09.972636 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:09.972639 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:09.972698 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 02:27:09.972738 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":126,"thread_start_us":102,"threads_started":1}
I20260925 02:27:09.973067 14047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b02320a3f86b430f84862acc1a0ec9c7" candidate_uuid: "01156eb3e7c3401c8f7c35234365615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3e97d413eb8491db9b232970ec8b95d" is_pre_election: true
I20260925 02:27:09.973130 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b02320a3f86b430f84862acc1a0ec9c7" candidate_uuid: "01156eb3e7c3401c8f7c35234365615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caf0a422dec44242bb926b891ef4d0df" is_pre_election: true
I20260925 02:27:09.973232 14047 raft_consensus.cc:2454] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01156eb3e7c3401c8f7c35234365615c in term 0.
I20260925 02:27:09.973276 14189 raft_consensus.cc:2454] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01156eb3e7c3401c8f7c35234365615c in term 0.
I20260925 02:27:09.973448 14262 leader_election.cc:302] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01156eb3e7c3401c8f7c35234365615c, b3e97d413eb8491db9b232970ec8b95d; no voters:
I20260925 02:27:09.973618 14403 raft_consensus.cc:2781] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:09.973651 14403 raft_consensus.cc:482] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:09.973665 14403 raft_consensus.cc:3037] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:09.974283 14403 raft_consensus.cc:504] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } }
I20260925 02:27:09.974416 14403 leader_election.cc:288] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [CANDIDATE]: Term 1 election: Requested vote from peers b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299), caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007)
I20260925 02:27:09.974545 14047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b02320a3f86b430f84862acc1a0ec9c7" candidate_uuid: "01156eb3e7c3401c8f7c35234365615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3e97d413eb8491db9b232970ec8b95d"
I20260925 02:27:09.974556 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b02320a3f86b430f84862acc1a0ec9c7" candidate_uuid: "01156eb3e7c3401c8f7c35234365615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caf0a422dec44242bb926b891ef4d0df"
I20260925 02:27:09.974619 14189 raft_consensus.cc:3037] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:09.974638 14047 raft_consensus.cc:3037] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:09.975211 14189 raft_consensus.cc:2454] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01156eb3e7c3401c8f7c35234365615c in term 1.
I20260925 02:27:09.975373 14260 leader_election.cc:302] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01156eb3e7c3401c8f7c35234365615c, caf0a422dec44242bb926b891ef4d0df; no voters:
I20260925 02:27:09.975383 14047 raft_consensus.cc:2454] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01156eb3e7c3401c8f7c35234365615c in term 1.
I20260925 02:27:09.975476 14403 raft_consensus.cc:2781] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:09.975625 14403 raft_consensus.cc:686] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 1 LEADER]: Becoming Leader. State: Replica: 01156eb3e7c3401c8f7c35234365615c, State: Running, Role: LEADER
I20260925 02:27:09.975723 14403 consensus_queue.cc:237] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } }
I20260925 02:27:09.976503 13888 catalog_manager.cc:5495] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01156eb3e7c3401c8f7c35234365615c (127.11.220.195). New cstate: current_term: 1 leader_uuid: "01156eb3e7c3401c8f7c35234365615c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } health_report { overall_health: UNKNOWN } } }
Sep 25 02:27:10 dist-test-slave-dn4n krb5kdc[13843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303229, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:10.015312 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.009036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:46803 (local address 127.0.0.1:57460)
0925 02:27:10.009212 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:10.009222 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:10.009321 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:10.009463 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:10.009464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:10.009744 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:10.009783 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.009785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.010410 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.010411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.010923 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.010925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.011286 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.011287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.011365 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.011888 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:10.011892 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:10.013267 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:10.014503 (+ 1236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:10.014509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:10.014510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:10.014674 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:10.014844 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:10.014844 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:10.014845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:10.014893 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:10.015069 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:10.015075 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:10.015118 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 02:27:10.015169 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:27:10.015248 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":97,"threads_started":1}
I20260925 02:27:10.015363 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.009130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57460 (local address 127.11.220.254:46803)
0925 02:27:10.009201 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:10.009203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:10.009307 (+ 104us) server_negotiation.cc:407] Connection header received
0925 02:27:10.009390 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:10.009391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:10.009399 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:10.009460 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:10.009862 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.010331 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.011020 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.011213 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.011287 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.013340 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:10.013342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:10.013342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:10.013348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:10.014443 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:10.014747 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:10.014748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:10.014748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:10.014792 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:10.014939 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:10.014940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:10.014940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:10.015024 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:10.015067 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:10.015192 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 02:27:10.015290 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":46}
I20260925 02:27:10.017318 13888 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:57460:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 02:27:10.017481 13888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:10.018880 14027 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf56022ccca4fbbbc107dcf01b08806 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7ebc33ef3d124910847b695c1aa172ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:27:10.018926 14169 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf56022ccca4fbbbc107dcf01b08806 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7ebc33ef3d124910847b695c1aa172ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:27:10.018921 14310 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf56022ccca4fbbbc107dcf01b08806 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7ebc33ef3d124910847b695c1aa172ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:27:10.019063 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf56022ccca4fbbbc107dcf01b08806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:10.019063 14027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf56022ccca4fbbbc107dcf01b08806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:10.019084 14310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf56022ccca4fbbbc107dcf01b08806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:10.020376 14397 tablet_bootstrap.cc:492] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: Bootstrap starting.
I20260925 02:27:10.020553 14398 tablet_bootstrap.cc:492] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: Bootstrap starting.
I20260925 02:27:10.020583 14402 tablet_bootstrap.cc:492] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: Bootstrap starting.
I20260925 02:27:10.020823 14397 tablet_bootstrap.cc:654] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:10.021121 14402 tablet_bootstrap.cc:654] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:10.021222 14398 tablet_bootstrap.cc:654] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:10.021538 14397 tablet_bootstrap.cc:492] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: No bootstrap required, opened a new log
I20260925 02:27:10.021581 14397 ts_tablet_manager.cc:1397] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:10.021732 14397 raft_consensus.cc:348] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.021816 14397 raft_consensus.cc:374] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:10.021817 14402 tablet_bootstrap.cc:492] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: No bootstrap required, opened a new log
I20260925 02:27:10.021835 14397 raft_consensus.cc:729] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01156eb3e7c3401c8f7c35234365615c, State: Initialized, Role: FOLLOWER
I20260925 02:27:10.021858 14402 ts_tablet_manager.cc:1397] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:10.021901 14397 consensus_queue.cc:260] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.021950 14398 tablet_bootstrap.cc:492] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: No bootstrap required, opened a new log
I20260925 02:27:10.021986 14398 ts_tablet_manager.cc:1397] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:10.021998 14397 ts_tablet_manager.cc:1428] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:10.022014 14402 raft_consensus.cc:348] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.022078 14402 raft_consensus.cc:374] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:10.022094 14402 raft_consensus.cc:729] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3e97d413eb8491db9b232970ec8b95d, State: Initialized, Role: FOLLOWER
I20260925 02:27:10.022130 14402 consensus_queue.cc:260] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.022119 14398 raft_consensus.cc:348] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.022187 14398 raft_consensus.cc:374] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:10.022207 14398 raft_consensus.cc:729] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf0a422dec44242bb926b891ef4d0df, State: Initialized, Role: FOLLOWER
I20260925 02:27:10.022217 14402 ts_tablet_manager.cc:1428] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:10.022244 14398 consensus_queue.cc:260] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.022325 14398 ts_tablet_manager.cc:1428] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:10.022275 14405 tablet_replica.cc:439] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } }
I20260925 02:27:10.022364 14405 tablet_replica.cc:442] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:10.022429 14403 tablet_replica.cc:439] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } }
I20260925 02:27:10.022501 14403 tablet_replica.cc:442] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:10.022645 14401 tablet_replica.cc:439] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } }
I20260925 02:27:10.022707 14401 tablet_replica.cc:442] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 02:27:10.056334 14095 tablet.cc:2369] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:10.056900 14401 raft_consensus.cc:482] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:10.056949 14401 raft_consensus.cc:504] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.057248 14401 leader_election.cc:288] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299), 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231)
I20260925 02:27:10.060572 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.057441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:56849 (local address 127.11.220.195:33231)
0925 02:27:10.057477 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:10.057478 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:10.057700 (+ 222us) server_negotiation.cc:407] Connection header received
0925 02:27:10.057835 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:10.057836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:10.057845 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:10.057894 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:10.058010 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.058598 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.059724 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.060235 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.060303 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.060311 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:10.060484 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 02:27:10.060515 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:27:10.060572 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.057429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:46127 (local address 127.11.220.193:35299)
0925 02:27:10.057470 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:10.057471 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:10.057480 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:10.057537 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:10.057538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:10.057547 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:10.057597 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:10.057855 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.058486 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.059724 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.060235 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.060303 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.060311 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:10.060477 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 02:27:10.060513 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:27:10.060600 14127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.057374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:35299 (local address 127.11.220.194:46127)
0925 02:27:10.057429 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:10.057439 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:10.057484 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:10.057620 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:10.057620 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:10.057622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:10.057693 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.057694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.058580 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.058581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.059652 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.059655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.060313 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.060314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.060398 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.060400 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:10.060471 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 02:27:10.060518 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":7}
I20260925 02:27:10.060609 14418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.057374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:33231 (local address 127.11.220.194:56849)
0925 02:27:10.057624 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:10.057633 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:10.057682 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:10.057907 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:10.057908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:10.057909 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:10.057955 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.057955 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.058651 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.058651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.059662 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.059664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.060336 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.060336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.060395 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.060399 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:10.060471 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:27:10.060518 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":112,"thread_start_us":92,"threads_started":1}
I20260925 02:27:10.060858 14047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf56022ccca4fbbbc107dcf01b08806" candidate_uuid: "caf0a422dec44242bb926b891ef4d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3e97d413eb8491db9b232970ec8b95d" is_pre_election: true
I20260925 02:27:10.060935 14047 raft_consensus.cc:2454] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caf0a422dec44242bb926b891ef4d0df in term 0.
I20260925 02:27:10.061097 14121 leader_election.cc:302] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3e97d413eb8491db9b232970ec8b95d, caf0a422dec44242bb926b891ef4d0df; no voters:
I20260925 02:27:10.061200 14401 raft_consensus.cc:2781] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:10.061164 14331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf56022ccca4fbbbc107dcf01b08806" candidate_uuid: "caf0a422dec44242bb926b891ef4d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01156eb3e7c3401c8f7c35234365615c" is_pre_election: true
I20260925 02:27:10.061249 14401 raft_consensus.cc:482] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:10.061271 14331 raft_consensus.cc:2454] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caf0a422dec44242bb926b891ef4d0df in term 0.
I20260925 02:27:10.061292 14401 raft_consensus.cc:3037] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:10.061854 14401 raft_consensus.cc:504] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.062002 14401 leader_election.cc:288] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [CANDIDATE]: Term 1 election: Requested vote from peers b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299), 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231)
I20260925 02:27:10.062135 14331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf56022ccca4fbbbc107dcf01b08806" candidate_uuid: "caf0a422dec44242bb926b891ef4d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01156eb3e7c3401c8f7c35234365615c"
I20260925 02:27:10.062143 14047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf56022ccca4fbbbc107dcf01b08806" candidate_uuid: "caf0a422dec44242bb926b891ef4d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3e97d413eb8491db9b232970ec8b95d"
I20260925 02:27:10.062201 14047 raft_consensus.cc:3037] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:10.062202 14331 raft_consensus.cc:3037] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:10.062870 14331 raft_consensus.cc:2454] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caf0a422dec44242bb926b891ef4d0df in term 1.
I20260925 02:27:10.062870 14047 raft_consensus.cc:2454] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caf0a422dec44242bb926b891ef4d0df in term 1.
I20260925 02:27:10.063030 14119 leader_election.cc:302] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01156eb3e7c3401c8f7c35234365615c, caf0a422dec44242bb926b891ef4d0df; no voters:
I20260925 02:27:10.063131 14401 raft_consensus.cc:2781] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:10.063267 14401 raft_consensus.cc:686] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [term 1 LEADER]: Becoming Leader. State: Replica: caf0a422dec44242bb926b891ef4d0df, State: Running, Role: LEADER
I20260925 02:27:10.063364 14401 consensus_queue.cc:237] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.063793 14419 tablet_replica.cc:439] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: TxnStatusTablet state changed. Reason: New leader caf0a422dec44242bb926b891ef4d0df. Latest consensus state: current_term: 1 leader_uuid: "caf0a422dec44242bb926b891ef4d0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } }
I20260925 02:27:10.063866 14419 tablet_replica.cc:442] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:27:10.064033 13888 catalog_manager.cc:5495] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df reported cstate change: term changed from 0 to 1, leader changed from <none> to caf0a422dec44242bb926b891ef4d0df (127.11.220.194). New cstate: current_term: 1 leader_uuid: "caf0a422dec44242bb926b891ef4d0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } health_report { overall_health: UNKNOWN } } }
I20260925 02:27:10.064306 14421 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 02:27:10.064415 14421 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 02:27:10.075317 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.068204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.11.220.194:42007)
0925 02:27:10.068259 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:10.068261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:10.068268 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:10.068314 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:10.068315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:10.068329 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:10.068400 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:10.068523 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.073923 (+ 5400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.074592 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.074765 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.074840 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.074997 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:10.075085 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:10.075174 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:10.075220 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 02:27:10.075239 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 02:27:10.075333 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.068114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:42007 (local address 127.0.0.1:59224)
0925 02:27:10.068176 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:10.068186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:10.068246 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:10.068408 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:10.068409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:10.068410 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:10.068468 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.068471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.074010 (+ 5539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.074011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.074514 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.074515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.074843 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.074845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.074903 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.074915 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:10.075155 (+ 240us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:10.075164 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 02:27:10.075215 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:27:10.075263 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 02:27:10.076829 14047 raft_consensus.cc:1261] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Refusing update from remote peer caf0a422dec44242bb926b891ef4d0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:10.076830 14331 raft_consensus.cc:1261] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Refusing update from remote peer caf0a422dec44242bb926b891ef4d0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:10.077117 14419 consensus_queue.cc:1035] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:10.077251 14401 consensus_queue.cc:1035] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:10.078447 14405 tablet_replica.cc:439] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: TxnStatusTablet state changed. Reason: New leader caf0a422dec44242bb926b891ef4d0df. Latest consensus state: current_term: 1 leader_uuid: "caf0a422dec44242bb926b891ef4d0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } }
I20260925 02:27:10.078554 14405 tablet_replica.cc:442] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:10.078508 14403 tablet_replica.cc:439] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: TxnStatusTablet state changed. Reason: New leader caf0a422dec44242bb926b891ef4d0df. Latest consensus state: current_term: 1 leader_uuid: "caf0a422dec44242bb926b891ef4d0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } }
I20260925 02:27:10.078593 14403 tablet_replica.cc:442] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:10.079779 14419 tablet_replica.cc:439] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "caf0a422dec44242bb926b891ef4d0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } }
I20260925 02:27:10.079851 14419 tablet_replica.cc:442] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:27:10.079939 14419 tablet_replica.cc:439] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "caf0a422dec44242bb926b891ef4d0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } }
I20260925 02:27:10.079975 14419 tablet_replica.cc:442] T 6bf56022ccca4fbbbc107dcf01b08806 P caf0a422dec44242bb926b891ef4d0df: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:27:10.080201 14424 mvcc.cc:205] Tried to move back new op lower bound from 7333082030392266752 to 7333082030340026368. Current Snapshot: MvccSnapshot[applied={T|T < 7333082030392266752}]
I20260925 02:27:10.080240 14405 tablet_replica.cc:439] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "caf0a422dec44242bb926b891ef4d0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } }
I20260925 02:27:10.080325 14405 tablet_replica.cc:442] T 6bf56022ccca4fbbbc107dcf01b08806 P b3e97d413eb8491db9b232970ec8b95d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:10.080406 14425 mvcc.cc:205] Tried to move back new op lower bound from 7333082030392266752 to 7333082030340026368. Current Snapshot: MvccSnapshot[applied={T|T < 7333082030392266752}]
I20260925 02:27:10.081023 14403 tablet_replica.cc:439] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "caf0a422dec44242bb926b891ef4d0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } } }
I20260925 02:27:10.081037 14426 mvcc.cc:205] Tried to move back new op lower bound from 7333082030392266752 to 7333082030340026368. Current Snapshot: MvccSnapshot[applied={T|T < 7333082030392266752}]
I20260925 02:27:10.081103 14403 tablet_replica.cc:442] T 6bf56022ccca4fbbbc107dcf01b08806 P 01156eb3e7c3401c8f7c35234365615c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:10.084965 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.082611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34468 (local address 127.11.220.195:33231)
0925 02:27:10.082644 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:10.082645 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:10.082670 (+ 25us) server_negotiation.cc:407] Connection header received
0925 02:27:10.082719 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:10.082719 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:10.082732 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:10.082797 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:10.082948 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.083398 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.084146 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.084344 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.084425 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.084655 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:10.084749 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:10.084836 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:10.084864 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 02:27:10.084884 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:27:10.089406 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.087221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:35929 (local address 127.11.220.194:42007)
0925 02:27:10.087252 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:10.087253 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:10.087282 (+ 29us) server_negotiation.cc:407] Connection header received
0925 02:27:10.087336 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:10.087337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:10.087344 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:10.087385 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:10.087486 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.087864 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.088853 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.089016 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.089066 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.089251 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:10.089271 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:10.089300 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:10.089353 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 02:27:10.089365 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 02:27:10.089454 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.087177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:42007 (local address 127.11.220.195:35929)
0925 02:27:10.087244 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:10.087253 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:10.087293 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:10.087390 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:10.087391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:10.087391 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:10.087439 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.087440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.087931 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.087932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.088765 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.088768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.089085 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.089086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.089153 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.089168 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:10.089304 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:10.089305 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 02:27:10.089351 (+ 46us) client_negotiation.cc:240] Negotiation successful
0925 02:27:10.089388 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 02:27:10.091457 14047 raft_consensus.cc:1261] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Refusing update from remote peer 01156eb3e7c3401c8f7c35234365615c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:10.091552 14189 raft_consensus.cc:1261] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Refusing update from remote peer 01156eb3e7c3401c8f7c35234365615c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:10.091684 14403 consensus_queue.cc:1035] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:10.091786 14407 consensus_queue.cc:1035] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [LEADER]: Connected to new peer: Peer: permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:10.099884 14241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.097500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:33231 (local address 127.11.220.194:59589)
0925 02:27:10.097585 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:10.097593 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:10.097625 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:10.097828 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:10.097829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:10.097830 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:10.097876 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.097877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.098392 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.098393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.099189 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:10.099191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.099584 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.099585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:10.099633 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.099646 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:10.099789 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:10.099790 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 02:27:10.099819 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 02:27:10.099843 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 02:27:10.099896 14395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:10.097694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:59589 (local address 127.11.220.195:33231)
0925 02:27:10.097742 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:10.097743 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:10.097749 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:27:10.097764 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:10.097764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:10.097771 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:10.097810 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:10.097933 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.098334 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.099272 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:10.099520 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:10.099576 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:10.099712 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:10.099739 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:10.099783 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:10.099821 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 02:27:10.099841 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:27:10.108697 13888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:10.108791 13888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:10.109462 13888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d: Sending DeleteTablet for 3 replicas of tablet b02320a3f86b430f84862acc1a0ec9c7
I20260925 02:27:10.109760 14027 tablet_service.cc:1514] Processing DeleteTablet for tablet b02320a3f86b430f84862acc1a0ec9c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:10 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:52646
I20260925 02:27:10.109889 14310 tablet_service.cc:1514] Processing DeleteTablet for tablet b02320a3f86b430f84862acc1a0ec9c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:10 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:34458
I20260925 02:27:10.109951 14169 tablet_service.cc:1514] Processing DeleteTablet for tablet b02320a3f86b430f84862acc1a0ec9c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:10 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:59222
I20260925 02:27:10.110137 14446 tablet_replica.cc:330] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: stopping tablet replica
I20260925 02:27:10.110217 14446 raft_consensus.cc:2229] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:10.110271 14446 raft_consensus.cc:2258] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:10.110364 13888 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:10.110486 13888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:10.110661 14446 ts_tablet_manager.cc:1905] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:10.111052 14444 tablet_replica.cc:330] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: stopping tablet replica
I20260925 02:27:10.111217 14444 raft_consensus.cc:2229] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:10.111320 14444 raft_consensus.cc:2258] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:10.111536 14445 tablet_replica.cc:330] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: stopping tablet replica
I20260925 02:27:10.111667 14445 raft_consensus.cc:2229] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:10.111770 14444 ts_tablet_manager.cc:1905] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:10.111811 14445 raft_consensus.cc:2258] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:10.112180 14027 tablet_service.cc:1467] Processing CreateTablet for tablet 10c20a888ede44cd8be7df746845636b (DEFAULT_TABLE table=test-table [id=d3d6cbae56374a0f88a3e060040c2c24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:10.112293 14027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10c20a888ede44cd8be7df746845636b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:10.112427 14445 ts_tablet_manager.cc:1905] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:10.112668 14169 tablet_service.cc:1467] Processing CreateTablet for tablet 10c20a888ede44cd8be7df746845636b (DEFAULT_TABLE table=test-table [id=d3d6cbae56374a0f88a3e060040c2c24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:10.112771 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10c20a888ede44cd8be7df746845636b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:10.113466 14444 ts_tablet_manager.cc:1918] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:10.113536 14444 log.cc:1196] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/b02320a3f86b430f84862acc1a0ec9c7
I20260925 02:27:10.113776 14444 ts_tablet_manager.cc:1939] T b02320a3f86b430f84862acc1a0ec9c7 P b3e97d413eb8491db9b232970ec8b95d: Deleting consensus metadata
I20260925 02:27:10.114096 14398 tablet_bootstrap.cc:492] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df: Bootstrap starting.
I20260925 02:27:10.114157 14310 tablet_service.cc:1467] Processing CreateTablet for tablet 10c20a888ede44cd8be7df746845636b (DEFAULT_TABLE table=test-table [id=d3d6cbae56374a0f88a3e060040c2c24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:10.114185 13874 catalog_manager.cc:4841] TS b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299): tablet b02320a3f86b430f84862acc1a0ec9c7 (table test-table [id=724e73da332147cb94cff4e10d1d43b9]) successfully deleted
I20260925 02:27:10.114238 14310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10c20a888ede44cd8be7df746845636b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:10.114470 14398 tablet_bootstrap.cc:654] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:10.115183 14445 ts_tablet_manager.cc:1918] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:10.115293 14445 log.cc:1196] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/b02320a3f86b430f84862acc1a0ec9c7
I20260925 02:27:10.115404 14397 tablet_bootstrap.cc:492] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c: Bootstrap starting.
I20260925 02:27:10.115648 14445 ts_tablet_manager.cc:1939] T b02320a3f86b430f84862acc1a0ec9c7 P 01156eb3e7c3401c8f7c35234365615c: Deleting consensus metadata
I20260925 02:27:10.116173 14397 tablet_bootstrap.cc:654] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:10.116391 14398 tablet_bootstrap.cc:492] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df: No bootstrap required, opened a new log
I20260925 02:27:10.116434 14398 ts_tablet_manager.cc:1397] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 02:27:10.116539 13872 catalog_manager.cc:4841] TS 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231): tablet b02320a3f86b430f84862acc1a0ec9c7 (table test-table [id=724e73da332147cb94cff4e10d1d43b9]) successfully deleted
I20260925 02:27:10.116575 14398 raft_consensus.cc:348] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.116631 14398 raft_consensus.cc:374] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:10.116647 14398 raft_consensus.cc:729] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf0a422dec44242bb926b891ef4d0df, State: Initialized, Role: FOLLOWER
I20260925 02:27:10.116678 14398 consensus_queue.cc:260] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.116750 14398 ts_tablet_manager.cc:1428] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:10.117624 14397 tablet_bootstrap.cc:492] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c: No bootstrap required, opened a new log
I20260925 02:27:10.117671 14397 ts_tablet_manager.cc:1397] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:10.117724 14402 tablet_bootstrap.cc:492] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d: Bootstrap starting.
I20260925 02:27:10.117805 14397 raft_consensus.cc:348] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.117875 14397 raft_consensus.cc:374] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:10.117894 14397 raft_consensus.cc:729] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01156eb3e7c3401c8f7c35234365615c, State: Initialized, Role: FOLLOWER
I20260925 02:27:10.117934 14397 consensus_queue.cc:260] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.118006 14397 ts_tablet_manager.cc:1428] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:10.118094 14402 tablet_bootstrap.cc:654] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:10.119629 14446 ts_tablet_manager.cc:1918] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:10.119663 14402 tablet_bootstrap.cc:492] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d: No bootstrap required, opened a new log
I20260925 02:27:10.119694 14446 log.cc:1196] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/b02320a3f86b430f84862acc1a0ec9c7
I20260925 02:27:10.119704 14402 ts_tablet_manager.cc:1397] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:10.119833 14402 raft_consensus.cc:348] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.119889 14402 raft_consensus.cc:374] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:10.119911 14402 raft_consensus.cc:729] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3e97d413eb8491db9b232970ec8b95d, State: Initialized, Role: FOLLOWER
I20260925 02:27:10.119958 14402 consensus_queue.cc:260] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.120061 14402 ts_tablet_manager.cc:1428] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:10.120867 14446 ts_tablet_manager.cc:1939] T b02320a3f86b430f84862acc1a0ec9c7 P caf0a422dec44242bb926b891ef4d0df: Deleting consensus metadata
I20260925 02:27:10.121207 13872 catalog_manager.cc:4841] TS caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007): tablet b02320a3f86b430f84862acc1a0ec9c7 (table test-table [id=724e73da332147cb94cff4e10d1d43b9]) successfully deleted
I20260925 02:27:10.146469 14440 raft_consensus.cc:482] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:10.146541 14440 raft_consensus.cc:504] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.146713 14440 leader_election.cc:288] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299), caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007)
I20260925 02:27:10.146821 14047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c20a888ede44cd8be7df746845636b" candidate_uuid: "01156eb3e7c3401c8f7c35234365615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3e97d413eb8491db9b232970ec8b95d" is_pre_election: true
I20260925 02:27:10.146888 14047 raft_consensus.cc:2454] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01156eb3e7c3401c8f7c35234365615c in term 0.
I20260925 02:27:10.146883 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c20a888ede44cd8be7df746845636b" candidate_uuid: "01156eb3e7c3401c8f7c35234365615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caf0a422dec44242bb926b891ef4d0df" is_pre_election: true
I20260925 02:27:10.146945 14189 raft_consensus.cc:2454] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01156eb3e7c3401c8f7c35234365615c in term 0.
I20260925 02:27:10.147010 14262 leader_election.cc:302] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01156eb3e7c3401c8f7c35234365615c, b3e97d413eb8491db9b232970ec8b95d; no voters:
I20260925 02:27:10.147080 14440 raft_consensus.cc:2781] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:10.147128 14440 raft_consensus.cc:482] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:10.147145 14440 raft_consensus.cc:3037] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:10.147704 14440 raft_consensus.cc:504] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.147820 14440 leader_election.cc:288] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [CANDIDATE]: Term 1 election: Requested vote from peers b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299), caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007)
I20260925 02:27:10.147979 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c20a888ede44cd8be7df746845636b" candidate_uuid: "01156eb3e7c3401c8f7c35234365615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caf0a422dec44242bb926b891ef4d0df"
I20260925 02:27:10.147982 14047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10c20a888ede44cd8be7df746845636b" candidate_uuid: "01156eb3e7c3401c8f7c35234365615c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3e97d413eb8491db9b232970ec8b95d"
I20260925 02:27:10.148029 14189 raft_consensus.cc:3037] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:10.148029 14047 raft_consensus.cc:3037] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:10.148639 14189 raft_consensus.cc:2454] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01156eb3e7c3401c8f7c35234365615c in term 1.
I20260925 02:27:10.148723 14047 raft_consensus.cc:2454] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01156eb3e7c3401c8f7c35234365615c in term 1.
I20260925 02:27:10.148778 14260 leader_election.cc:302] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01156eb3e7c3401c8f7c35234365615c, caf0a422dec44242bb926b891ef4d0df; no voters:
I20260925 02:27:10.148880 14440 raft_consensus.cc:2781] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:10.148918 14440 raft_consensus.cc:686] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 1 LEADER]: Becoming Leader. State: Replica: 01156eb3e7c3401c8f7c35234365615c, State: Running, Role: LEADER
I20260925 02:27:10.148962 14440 consensus_queue.cc:237] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:10.149384 13888 catalog_manager.cc:5495] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01156eb3e7c3401c8f7c35234365615c (127.11.220.195). New cstate: current_term: 1 leader_uuid: "01156eb3e7c3401c8f7c35234365615c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } health_report { overall_health: HEALTHY } } }
I20260925 02:27:10.156695 14046 raft_consensus.cc:1261] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Refusing update from remote peer 01156eb3e7c3401c8f7c35234365615c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:10.156870 14407 consensus_queue.cc:1035] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:10.157080 14189 raft_consensus.cc:1261] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Refusing update from remote peer 01156eb3e7c3401c8f7c35234365615c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:10.157366 14407 consensus_queue.cc:1035] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [LEADER]: Connected to new peer: Peer: permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 02:27:10.162329 14426 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 02:27:11.679229 13888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:11.679327 13888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:11.680094 13888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d: Sending DeleteTablet for 3 replicas of tablet 10c20a888ede44cd8be7df746845636b
I20260925 02:27:11.680322 14027 tablet_service.cc:1514] Processing DeleteTablet for tablet 10c20a888ede44cd8be7df746845636b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:11 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:52646
I20260925 02:27:11.680370 14310 tablet_service.cc:1514] Processing DeleteTablet for tablet 10c20a888ede44cd8be7df746845636b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:11 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:34458
I20260925 02:27:11.680516 14168 tablet_service.cc:1514] Processing DeleteTablet for tablet 10c20a888ede44cd8be7df746845636b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:11 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:59222
I20260925 02:27:11.680688 14476 tablet_replica.cc:330] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df: stopping tablet replica
I20260925 02:27:11.680747 14476 raft_consensus.cc:2229] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:11.680790 14476 raft_consensus.cc:2258] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:11.682376 14476 ts_tablet_manager.cc:1905] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:11.682519 14474 tablet_replica.cc:330] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d: stopping tablet replica
I20260925 02:27:11.682615 14474 raft_consensus.cc:2229] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:11.682699 14474 raft_consensus.cc:2258] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:11.682821 14475 tablet_replica.cc:330] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c: stopping tablet replica
I20260925 02:27:11.682873 14475 raft_consensus.cc:2229] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:11.682926 14475 raft_consensus.cc:2258] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:11.683094 14475 ts_tablet_manager.cc:1905] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:11.683094 14474 ts_tablet_manager.cc:1905] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:11.684629 14474 ts_tablet_manager.cc:1918] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 02:27:11.684629 14475 ts_tablet_manager.cc:1918] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 02:27:11.684697 14474 log.cc:1196] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/10c20a888ede44cd8be7df746845636b
I20260925 02:27:11.684793 14475 log.cc:1196] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/10c20a888ede44cd8be7df746845636b
I20260925 02:27:11.684903 14474 ts_tablet_manager.cc:1939] T 10c20a888ede44cd8be7df746845636b P b3e97d413eb8491db9b232970ec8b95d: Deleting consensus metadata
I20260925 02:27:11.685060 14476 ts_tablet_manager.cc:1918] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 02:27:11.685119 14476 log.cc:1196] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/10c20a888ede44cd8be7df746845636b
I20260925 02:27:11.685094 14475 ts_tablet_manager.cc:1939] T 10c20a888ede44cd8be7df746845636b P 01156eb3e7c3401c8f7c35234365615c: Deleting consensus metadata
I20260925 02:27:11.685345 14476 ts_tablet_manager.cc:1939] T 10c20a888ede44cd8be7df746845636b P caf0a422dec44242bb926b891ef4d0df: Deleting consensus metadata
I20260925 02:27:11.685377 13874 catalog_manager.cc:4841] TS b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299): tablet 10c20a888ede44cd8be7df746845636b (table test-table [id=d3d6cbae56374a0f88a3e060040c2c24]) successfully deleted
I20260925 02:27:11.685551 13872 catalog_manager.cc:4841] TS 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231): tablet 10c20a888ede44cd8be7df746845636b (table test-table [id=d3d6cbae56374a0f88a3e060040c2c24]) successfully deleted
I20260925 02:27:11.685618 13872 catalog_manager.cc:4841] TS caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007): tablet 10c20a888ede44cd8be7df746845636b (table test-table [id=d3d6cbae56374a0f88a3e060040c2c24]) successfully deleted
Sep 25 02:27:11 dist-test-slave-dn4n krb5kdc[13843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303231, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 02:27:11 dist-test-slave-dn4n krb5kdc[13843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303231, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:11.711172 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.696676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57468 (local address 127.11.220.254:46803)
0925 02:27:11.696805 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:11.696807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:11.696815 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:11.696833 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:11.696833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:11.696842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:11.696894 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:11.697380 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.697839 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.698831 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.698992 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.699032 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.708920 (+ 9888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:11.708922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:11.708922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:11.708929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:11.710036 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:11.710379 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:11.710380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:11.710381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:11.710456 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:11.710654 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:11.710655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:11.710655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:11.710766 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:11.710869 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:11.710978 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 02:27:11.711060 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260925 02:27:11.712608 13888 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:11.712739 13888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:11.713958 14027 tablet_service.cc:1467] Processing CreateTablet for tablet 85aecb615eba4d1592cbde2ccca85098 (DEFAULT_TABLE table=test-table [id=b94d43f9f485460a9cccf13147662b96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:11.713979 14310 tablet_service.cc:1467] Processing CreateTablet for tablet 85aecb615eba4d1592cbde2ccca85098 (DEFAULT_TABLE table=test-table [id=b94d43f9f485460a9cccf13147662b96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:11.714084 14027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85aecb615eba4d1592cbde2ccca85098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:11.714085 14310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85aecb615eba4d1592cbde2ccca85098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:11.714931 14485 tablet_bootstrap.cc:492] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d: Bootstrap starting.
I20260925 02:27:11.714933 14486 tablet_bootstrap.cc:492] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c: Bootstrap starting.
I20260925 02:27:11.714969 14168 tablet_service.cc:1467] Processing CreateTablet for tablet 85aecb615eba4d1592cbde2ccca85098 (DEFAULT_TABLE table=test-table [id=b94d43f9f485460a9cccf13147662b96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:11.715071 14168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85aecb615eba4d1592cbde2ccca85098. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:11.715286 14486 tablet_bootstrap.cc:654] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:11.715379 14485 tablet_bootstrap.cc:654] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:11.716120 14485 tablet_bootstrap.cc:492] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d: No bootstrap required, opened a new log
I20260925 02:27:11.716178 14485 ts_tablet_manager.cc:1397] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:11.716326 14485 raft_consensus.cc:348] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.716409 14485 raft_consensus.cc:374] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:11.716430 14485 raft_consensus.cc:729] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3e97d413eb8491db9b232970ec8b95d, State: Initialized, Role: FOLLOWER
I20260925 02:27:11.716523 14485 consensus_queue.cc:260] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.716562 14489 tablet_bootstrap.cc:492] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df: Bootstrap starting.
I20260925 02:27:11.716702 14485 ts_tablet_manager.cc:1428] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:11.717018 14489 tablet_bootstrap.cc:654] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:11.717065 14486 tablet_bootstrap.cc:492] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c: No bootstrap required, opened a new log
I20260925 02:27:11.717125 14486 ts_tablet_manager.cc:1397] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:11.717243 14486 raft_consensus.cc:348] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.717327 14486 raft_consensus.cc:374] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:11.717350 14486 raft_consensus.cc:729] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01156eb3e7c3401c8f7c35234365615c, State: Initialized, Role: FOLLOWER
I20260925 02:27:11.717398 14486 consensus_queue.cc:260] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.717489 14486 ts_tablet_manager.cc:1428] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:11.717573 14489 tablet_bootstrap.cc:492] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df: No bootstrap required, opened a new log
I20260925 02:27:11.717638 14489 ts_tablet_manager.cc:1397] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:11.717737 14489 raft_consensus.cc:348] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.717797 14489 raft_consensus.cc:374] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:11.717814 14489 raft_consensus.cc:729] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf0a422dec44242bb926b891ef4d0df, State: Initialized, Role: FOLLOWER
I20260925 02:27:11.717849 14489 consensus_queue.cc:260] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.718042 14489 ts_tablet_manager.cc:1428] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:11.725147 14490 raft_consensus.cc:482] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:11.725195 14490 raft_consensus.cc:504] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.725514 14490 leader_election.cc:288] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007), 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231)
I20260925 02:27:11.728895 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.725649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:41853 (local address 127.11.220.194:42007)
0925 02:27:11.725767 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:11.725769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:11.725801 (+ 32us) server_negotiation.cc:407] Connection header received
0925 02:27:11.725883 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:11.725883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:11.725893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:11.725952 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:11.726054 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.726706 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.728030 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.728580 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.728636 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.728643 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:11.728799 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 02:27:11.728837 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260925 02:27:11.728895 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.725850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:44371 (local address 127.11.220.195:33231)
0925 02:27:11.725986 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:11.725987 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:11.725993 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:27:11.726010 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:11.726010 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:11.726018 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:11.726073 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:11.726179 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.726770 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.728028 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.728564 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.728639 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.728646 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:11.728799 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 02:27:11.728839 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260925 02:27:11.728951 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.725613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:42007 (local address 127.11.220.193:41853)
0925 02:27:11.725754 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:11.725765 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:11.725817 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:11.725957 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:11.725958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:11.725960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:11.726009 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:11.726010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.726768 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.726769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:11.727938 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:11.727940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.728648 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.728649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:11.728728 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.728733 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:27:11.728784 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:27:11.728829 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":50,"thread_start_us":87,"threads_started":1}
I20260925 02:27:11.728951 14494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.725809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:33231 (local address 127.11.220.193:44371)
0925 02:27:11.725910 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:11.725918 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:11.725959 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:11.726078 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:11.726079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:11.726080 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:11.726126 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:11.726127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.726846 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.726847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:11.727938 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:11.727940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.728650 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.728651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:11.728731 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.728733 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:11.728784 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:27:11.728829 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":9,"thread_start_us":55,"threads_started":1}
I20260925 02:27:11.729215 14331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85aecb615eba4d1592cbde2ccca85098" candidate_uuid: "b3e97d413eb8491db9b232970ec8b95d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01156eb3e7c3401c8f7c35234365615c" is_pre_election: true
I20260925 02:27:11.729290 14331 raft_consensus.cc:2454] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3e97d413eb8491db9b232970ec8b95d in term 0.
I20260925 02:27:11.729470 13977 leader_election.cc:302] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01156eb3e7c3401c8f7c35234365615c, b3e97d413eb8491db9b232970ec8b95d; no voters:
I20260925 02:27:11.729519 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85aecb615eba4d1592cbde2ccca85098" candidate_uuid: "b3e97d413eb8491db9b232970ec8b95d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caf0a422dec44242bb926b891ef4d0df" is_pre_election: true
I20260925 02:27:11.729578 14189 raft_consensus.cc:2454] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3e97d413eb8491db9b232970ec8b95d in term 0.
I20260925 02:27:11.729609 14490 raft_consensus.cc:2781] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:11.729656 14490 raft_consensus.cc:482] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:11.729681 14490 raft_consensus.cc:3037] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:11.730264 14490 raft_consensus.cc:504] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.730346 14490 leader_election.cc:288] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [CANDIDATE]: Term 1 election: Requested vote from peers caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007), 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231)
I20260925 02:27:11.730494 14189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85aecb615eba4d1592cbde2ccca85098" candidate_uuid: "b3e97d413eb8491db9b232970ec8b95d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caf0a422dec44242bb926b891ef4d0df"
I20260925 02:27:11.730499 14331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85aecb615eba4d1592cbde2ccca85098" candidate_uuid: "b3e97d413eb8491db9b232970ec8b95d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01156eb3e7c3401c8f7c35234365615c"
I20260925 02:27:11.730569 14189 raft_consensus.cc:3037] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:11.730567 14331 raft_consensus.cc:3037] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:11.731001 14189 raft_consensus.cc:2454] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3e97d413eb8491db9b232970ec8b95d in term 1.
I20260925 02:27:11.731041 14331 raft_consensus.cc:2454] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3e97d413eb8491db9b232970ec8b95d in term 1.
I20260925 02:27:11.731180 13977 leader_election.cc:302] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3e97d413eb8491db9b232970ec8b95d, caf0a422dec44242bb926b891ef4d0df; no voters:
I20260925 02:27:11.731261 14490 raft_consensus.cc:2781] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:11.731362 14490 raft_consensus.cc:686] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 1 LEADER]: Becoming Leader. State: Replica: b3e97d413eb8491db9b232970ec8b95d, State: Running, Role: LEADER
I20260925 02:27:11.731429 14490 consensus_queue.cc:237] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.731977 13888 catalog_manager.cc:5495] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d reported cstate change: term changed from 0 to 1, leader changed from <none> to b3e97d413eb8491db9b232970ec8b95d (127.11.220.193). New cstate: current_term: 1 leader_uuid: "b3e97d413eb8491db9b232970ec8b95d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } health_report { overall_health: UNKNOWN } } }
I20260925 02:27:11.739624 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.737699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52650 (local address 127.11.220.193:35299)
0925 02:27:11.737813 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:11.737815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:11.737822 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:11.737857 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:11.737858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:11.737872 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:11.737933 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:11.737994 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.738428 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.739038 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.739224 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.739267 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.739390 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:11.739475 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:11.739523 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:11.739552 (+ 29us) server_negotiation.cc:299] Negotiation successful
0925 02:27:11.739565 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":77,"threads_started":1}
I20260925 02:27:11.743412 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.741632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:46601 (local address 127.11.220.194:42007)
0925 02:27:11.741682 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:11.741683 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:11.741783 (+ 100us) server_negotiation.cc:407] Connection header received
0925 02:27:11.741834 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:11.741835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:11.741841 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:11.741886 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:11.741967 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.742341 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.742954 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.743107 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.743148 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.743279 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:11.743300 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:11.743333 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:11.743360 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 02:27:11.743373 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 02:27:11.743434 14502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.741596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:42007 (local address 127.11.220.193:46601)
0925 02:27:11.741734 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:11.741742 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:11.741793 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:11.741895 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:11.741895 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:11.741896 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:11.741934 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:11.741935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.742390 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.742391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:11.742891 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:11.742892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.743162 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.743163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:11.743208 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.743220 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:11.743332 (+ 112us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:11.743333 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 02:27:11.743359 (+ 26us) client_negotiation.cc:240] Negotiation successful
0925 02:27:11.743384 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260925 02:27:11.745129 14189 raft_consensus.cc:1261] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Refusing update from remote peer b3e97d413eb8491db9b232970ec8b95d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:11.745182 14331 raft_consensus.cc:1261] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Refusing update from remote peer b3e97d413eb8491db9b232970ec8b95d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:11.745339 14496 consensus_queue.cc:1035] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [LEADER]: Connected to new peer: Peer: permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:11.745466 14490 consensus_queue.cc:1035] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [LEADER]: Connected to new peer: Peer: permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:11.751338 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.749375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:40353 (local address 127.11.220.193:35299)
0925 02:27:11.749411 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:11.749413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:11.749490 (+ 77us) server_negotiation.cc:407] Connection header received
0925 02:27:11.749541 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:11.749542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:11.749548 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:11.749578 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:11.749636 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.749976 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.750676 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.750945 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.750999 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.751138 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:11.751166 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:11.751215 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:11.751267 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 02:27:11.751283 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:27:11.751389 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.749347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:35299 (local address 127.11.220.194:40353)
0925 02:27:11.749446 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:11.749453 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:11.749502 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:11.749580 (+ 78us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:11.749580 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:11.749581 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:11.749608 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:11.749608 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.750024 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.750024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:11.750610 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:11.750612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.751009 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.751010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:11.751066 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.751071 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:11.751222 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:11.751224 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:11.751265 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 02:27:11.751309 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":64,"threads_started":1}
I20260925 02:27:11.757285 13886 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:11.757397 13886 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:11.757906 13886 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d: Sending DeleteTablet for 3 replicas of tablet 85aecb615eba4d1592cbde2ccca85098
I20260925 02:27:11.758122 14310 tablet_service.cc:1514] Processing DeleteTablet for tablet 85aecb615eba4d1592cbde2ccca85098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:11 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:34458
I20260925 02:27:11.758127 14027 tablet_service.cc:1514] Processing DeleteTablet for tablet 85aecb615eba4d1592cbde2ccca85098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:11 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:52646
I20260925 02:27:11.758193 14474 tablet_replica.cc:330] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d: stopping tablet replica
I20260925 02:27:11.758195 14475 tablet_replica.cc:330] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c: stopping tablet replica
I20260925 02:27:11.758234 14474 raft_consensus.cc:2229] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:11.758237 14475 raft_consensus.cc:2229] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:11.758271 14475 raft_consensus.cc:2258] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:11.758301 14474 raft_consensus.cc:2258] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:11.758373 13886 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:11.758497 13886 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:11.758663 14474 ts_tablet_manager.cc:1905] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:11.759579 14310 tablet_service.cc:1467] Processing CreateTablet for tablet 21ab3aad97a94bf289a9feb5bd638451 (DEFAULT_TABLE table=test-table [id=05f8e87ffd934435990ff1642da67f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:11.759584 14169 tablet_service.cc:1467] Processing CreateTablet for tablet 21ab3aad97a94bf289a9feb5bd638451 (DEFAULT_TABLE table=test-table [id=05f8e87ffd934435990ff1642da67f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:11.759682 14310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ab3aad97a94bf289a9feb5bd638451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:11.759690 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ab3aad97a94bf289a9feb5bd638451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:11.760109 14168 tablet_service.cc:1514] Processing DeleteTablet for tablet 85aecb615eba4d1592cbde2ccca85098 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:11 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:59222
I20260925 02:27:11.760275 14027 tablet_service.cc:1467] Processing CreateTablet for tablet 21ab3aad97a94bf289a9feb5bd638451 (DEFAULT_TABLE table=test-table [id=05f8e87ffd934435990ff1642da67f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:11.760519 14476 tablet_replica.cc:330] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df: stopping tablet replica
I20260925 02:27:11.760529 14475 ts_tablet_manager.cc:1905] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:11.760583 14476 raft_consensus.cc:2229] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:11.760614 14476 raft_consensus.cc:2258] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:11.760936 14476 ts_tablet_manager.cc:1905] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:11.761794 14475 ts_tablet_manager.cc:1918] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:11.761849 14475 log.cc:1196] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/85aecb615eba4d1592cbde2ccca85098
I20260925 02:27:11.761953 14476 ts_tablet_manager.cc:1918] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:11.762024 14476 log.cc:1196] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/85aecb615eba4d1592cbde2ccca85098
I20260925 02:27:11.762037 14475 ts_tablet_manager.cc:1939] T 85aecb615eba4d1592cbde2ccca85098 P 01156eb3e7c3401c8f7c35234365615c: Deleting consensus metadata
I20260925 02:27:11.762269 14486 tablet_bootstrap.cc:492] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c: Bootstrap starting.
I20260925 02:27:11.762274 14476 ts_tablet_manager.cc:1939] T 85aecb615eba4d1592cbde2ccca85098 P caf0a422dec44242bb926b891ef4d0df: Deleting consensus metadata
I20260925 02:27:11.762538 13872 catalog_manager.cc:4841] TS 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231): tablet 85aecb615eba4d1592cbde2ccca85098 (table test-table [id=b94d43f9f485460a9cccf13147662b96]) successfully deleted
I20260925 02:27:11.762554 14486 tablet_bootstrap.cc:654] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:11.762599 13872 catalog_manager.cc:4841] TS caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007): tablet 85aecb615eba4d1592cbde2ccca85098 (table test-table [id=b94d43f9f485460a9cccf13147662b96]) successfully deleted
I20260925 02:27:11.763067 14489 tablet_bootstrap.cc:492] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df: Bootstrap starting.
I20260925 02:27:11.763360 14474 ts_tablet_manager.cc:1918] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:11.763391 14489 tablet_bootstrap.cc:654] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:11.763386 14027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ab3aad97a94bf289a9feb5bd638451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:11.763408 14474 log.cc:1196] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/85aecb615eba4d1592cbde2ccca85098
I20260925 02:27:11.763614 14474 ts_tablet_manager.cc:1939] T 85aecb615eba4d1592cbde2ccca85098 P b3e97d413eb8491db9b232970ec8b95d: Deleting consensus metadata
I20260925 02:27:11.763859 14489 tablet_bootstrap.cc:492] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df: No bootstrap required, opened a new log
I20260925 02:27:11.763885 13874 catalog_manager.cc:4841] TS b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299): tablet 85aecb615eba4d1592cbde2ccca85098 (table test-table [id=b94d43f9f485460a9cccf13147662b96]) successfully deleted
I20260925 02:27:11.763897 14489 ts_tablet_manager.cc:1397] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:11.763990 14489 raft_consensus.cc:348] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.764055 14489 raft_consensus.cc:374] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:11.764077 14489 raft_consensus.cc:729] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf0a422dec44242bb926b891ef4d0df, State: Initialized, Role: FOLLOWER
I20260925 02:27:11.764120 14489 consensus_queue.cc:260] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.764211 14489 ts_tablet_manager.cc:1428] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:11.764482 14485 tablet_bootstrap.cc:492] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d: Bootstrap starting.
I20260925 02:27:11.764644 14486 tablet_bootstrap.cc:492] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c: No bootstrap required, opened a new log
I20260925 02:27:11.764685 14486 ts_tablet_manager.cc:1397] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260925 02:27:11.764770 14486 raft_consensus.cc:348] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.764819 14486 raft_consensus.cc:374] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:11.764838 14486 raft_consensus.cc:729] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01156eb3e7c3401c8f7c35234365615c, State: Initialized, Role: FOLLOWER
I20260925 02:27:11.764860 14485 tablet_bootstrap.cc:654] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:11.764876 14486 consensus_queue.cc:260] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.764950 14486 ts_tablet_manager.cc:1428] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:11.765439 14485 tablet_bootstrap.cc:492] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d: No bootstrap required, opened a new log
I20260925 02:27:11.765497 14485 ts_tablet_manager.cc:1397] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 02:27:11.765633 14485 raft_consensus.cc:348] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.765698 14485 raft_consensus.cc:374] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:11.765719 14485 raft_consensus.cc:729] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3e97d413eb8491db9b232970ec8b95d, State: Initialized, Role: FOLLOWER
I20260925 02:27:11.765761 14485 consensus_queue.cc:260] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.765856 14485 ts_tablet_manager.cc:1428] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:11.781468 14456 raft_consensus.cc:482] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:11.781529 14456 raft_consensus.cc:504] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.781678 14456 leader_election.cc:288] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299), 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231)
I20260925 02:27:11.781803 14047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ab3aad97a94bf289a9feb5bd638451" candidate_uuid: "caf0a422dec44242bb926b891ef4d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3e97d413eb8491db9b232970ec8b95d" is_pre_election: true
I20260925 02:27:11.781850 14331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ab3aad97a94bf289a9feb5bd638451" candidate_uuid: "caf0a422dec44242bb926b891ef4d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01156eb3e7c3401c8f7c35234365615c" is_pre_election: true
I20260925 02:27:11.781891 14047 raft_consensus.cc:2454] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caf0a422dec44242bb926b891ef4d0df in term 0.
I20260925 02:27:11.781936 14331 raft_consensus.cc:2454] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caf0a422dec44242bb926b891ef4d0df in term 0.
I20260925 02:27:11.782047 14121 leader_election.cc:302] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3e97d413eb8491db9b232970ec8b95d, caf0a422dec44242bb926b891ef4d0df; no voters:
I20260925 02:27:11.782133 14456 raft_consensus.cc:2781] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:11.782163 14456 raft_consensus.cc:482] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:11.782196 14456 raft_consensus.cc:3037] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:11.782701 14456 raft_consensus.cc:504] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.782850 14456 leader_election.cc:288] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [CANDIDATE]: Term 1 election: Requested vote from peers b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299), 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231)
I20260925 02:27:11.782969 14331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ab3aad97a94bf289a9feb5bd638451" candidate_uuid: "caf0a422dec44242bb926b891ef4d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01156eb3e7c3401c8f7c35234365615c"
I20260925 02:27:11.783001 14047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ab3aad97a94bf289a9feb5bd638451" candidate_uuid: "caf0a422dec44242bb926b891ef4d0df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3e97d413eb8491db9b232970ec8b95d"
I20260925 02:27:11.783034 14331 raft_consensus.cc:3037] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:11.783061 14047 raft_consensus.cc:3037] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:11.783602 14047 raft_consensus.cc:2454] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caf0a422dec44242bb926b891ef4d0df in term 1.
I20260925 02:27:11.783658 14331 raft_consensus.cc:2454] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caf0a422dec44242bb926b891ef4d0df in term 1.
I20260925 02:27:11.783780 14121 leader_election.cc:302] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3e97d413eb8491db9b232970ec8b95d, caf0a422dec44242bb926b891ef4d0df; no voters:
I20260925 02:27:11.783857 14456 raft_consensus.cc:2781] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:11.783917 14456 raft_consensus.cc:686] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 1 LEADER]: Becoming Leader. State: Replica: caf0a422dec44242bb926b891ef4d0df, State: Running, Role: LEADER
I20260925 02:27:11.783959 14456 consensus_queue.cc:237] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } }
I20260925 02:27:11.784432 13886 catalog_manager.cc:5495] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df reported cstate change: term changed from 0 to 1, leader changed from <none> to caf0a422dec44242bb926b891ef4d0df (127.11.220.194). New cstate: current_term: 1 leader_uuid: "caf0a422dec44242bb926b891ef4d0df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caf0a422dec44242bb926b891ef4d0df" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 } health_report { overall_health: UNKNOWN } } }
I20260925 02:27:11.794380 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.791993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59226 (local address 127.11.220.194:42007)
0925 02:27:11.792028 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:11.792030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:11.792051 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:27:11.792139 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:11.792140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:11.792149 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:11.792221 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:11.792344 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.792867 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.793848 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.794004 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.794044 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.794207 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:11.794229 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:11.794255 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:11.794325 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 02:27:11.794340 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:11.798162 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.796080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:42007 (local address 127.11.220.194:38085)
0925 02:27:11.796144 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:11.796153 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:11.796205 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:11.796355 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:11.796356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:11.796357 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:11.796384 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:11.796385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.797096 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.797096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:11.797522 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:11.797523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.797905 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.797905 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:11.797936 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.797941 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:11.798046 (+ 105us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:11.798047 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 02:27:11.798072 (+ 25us) client_negotiation.cc:240] Negotiation successful
0925 02:27:11.798095 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 02:27:11.798177 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:11.796162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:38085 (local address 127.11.220.194:42007)
0925 02:27:11.796187 (+ 25us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:11.796190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:11.796197 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:11.796273 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:11.796274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:11.796282 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:11.796328 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:11.796426 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.797035 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.797590 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:11.797840 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:11.797900 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:11.797979 (+ 79us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:11.798003 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:11.798046 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:11.798074 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 02:27:11.798087 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260925 02:27:11.799729 14331 raft_consensus.cc:1261] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Refusing update from remote peer caf0a422dec44242bb926b891ef4d0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:11.799794 14047 raft_consensus.cc:1261] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Refusing update from remote peer caf0a422dec44242bb926b891ef4d0df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:11.799938 14461 consensus_queue.cc:1035] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "01156eb3e7c3401c8f7c35234365615c" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:11.800122 14456 consensus_queue.cc:1035] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3e97d413eb8491db9b232970ec8b95d" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 35299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 02:27:11.804180 14454 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 02:27:13.323617 13888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:13.323710 13888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:13.324484 13888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e9e64fb415d474fa97f5149ce22686d: Sending DeleteTablet for 3 replicas of tablet 21ab3aad97a94bf289a9feb5bd638451
I20260925 02:27:13.324712 14027 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ab3aad97a94bf289a9feb5bd638451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:13 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:52646
I20260925 02:27:13.324766 14310 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ab3aad97a94bf289a9feb5bd638451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:13 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:34458
I20260925 02:27:13.324888 14168 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ab3aad97a94bf289a9feb5bd638451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:13 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:59222
I20260925 02:27:13.324983 14538 tablet_replica.cc:330] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c: stopping tablet replica
I20260925 02:27:13.325047 14538 raft_consensus.cc:2229] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:13.325059 14537 tablet_replica.cc:330] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d: stopping tablet replica
I20260925 02:27:13.325089 14538 raft_consensus.cc:2258] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:13.325101 14537 raft_consensus.cc:2229] T 21ab3aad97a94bf289a9feb5bd638451 P b3e97d413eb8491db9b232970ec8b95d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:13.325088 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 13960
I20260925 02:27:13.325191 14539 tablet_replica.cc:330] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df: stopping tablet replica
I20260925 02:27:13.325274 14539 raft_consensus.cc:2229] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:13.325531 14539 raft_consensus.cc:2258] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:13.325836 14539 ts_tablet_manager.cc:1905] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:13.327999 14539 ts_tablet_manager.cc:1918] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 02:27:13.328082 14539 log.cc:1196] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/21ab3aad97a94bf289a9feb5bd638451
I20260925 02:27:13.328323 14539 ts_tablet_manager.cc:1939] T 21ab3aad97a94bf289a9feb5bd638451 P caf0a422dec44242bb926b891ef4d0df: Deleting consensus metadata
I20260925 02:27:13.329439 13872 catalog_manager.cc:4841] TS caf0a422dec44242bb926b891ef4d0df (127.11.220.194:42007): tablet 21ab3aad97a94bf289a9feb5bd638451 (table test-table [id=05f8e87ffd934435990ff1642da67f59]) successfully deleted
W20260925 02:27:13.332340 13874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:35299 (error 108)
I20260925 02:27:13.332574 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 14103
I20260925 02:27:13.333786 14538 ts_tablet_manager.cc:1905] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:13.335008 14541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:13.333492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:35299 (local address 127.0.0.1:52652)
0925 02:27:13.334939 (+ 1447us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:13.334979 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:35299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1425,"thread_start_us":54,"threads_started":1}
W20260925 02:27:13.335124 13874 catalog_manager.cc:4584] TS b3e97d413eb8491db9b232970ec8b95d (127.11.220.193:35299): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21ab3aad97a94bf289a9feb5bd638451: Network error: Client connection negotiation failed: client connection to 127.11.220.193:35299: connect: Connection refused (error 111)
I20260925 02:27:13.335330 14538 ts_tablet_manager.cc:1918] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 02:27:13.335407 14538 log.cc:1196] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/21ab3aad97a94bf289a9feb5bd638451
I20260925 02:27:13.335620 14538 ts_tablet_manager.cc:1939] T 21ab3aad97a94bf289a9feb5bd638451 P 01156eb3e7c3401c8f7c35234365615c: Deleting consensus metadata
I20260925 02:27:13.335934 13872 catalog_manager.cc:4841] TS 01156eb3e7c3401c8f7c35234365615c (127.11.220.195:33231): tablet 21ab3aad97a94bf289a9feb5bd638451 (table test-table [id=05f8e87ffd934435990ff1642da67f59]) successfully deleted
I20260925 02:27:13.339746 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 14244
I20260925 02:27:13.345095 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 13856
2026-09-25T02:27:13Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5142 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:13 dist-test-slave-dn4n krb5kdc[14544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:13 dist-test-slave-dn4n krb5kdc[14544](info): set up 2 sockets
Sep 25 02:27:13 dist-test-slave-dn4n krb5kdc[14544](info): commencing operation
krb5kdc: starting...
W20260925 02:27:14.284946 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:14 dist-test-slave-dn4n krb5kdc[14544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:14.384724 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:33607
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:33607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:14.455231 14557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:14.455422 14557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:14.455457 14557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:14.456475 14557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:14.456534 14557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:14.456558 14557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:14.456588 14557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:14.456616 14557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:14.457928 14557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:33607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:33607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:14.458171 14557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:14.458369 14557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:14.462602 14565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:14.462605 14562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:14.462639 14557 server_base.cc:1034] running on GCE node
W20260925 02:27:14.462608 14563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:14.463073 14557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:14.463322 14557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:14.464490 14557 hybrid_clock.cc:648] HybridClock initialized: now 1790303234464482 us; error 33 us; skew 500 ppm
Sep 25 02:27:14 dist-test-slave-dn4n krb5kdc[14544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:14.466583 14557 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:14.466926 14557 webserver.cc:460] Webserver started at http://127.11.220.254:40365/ using document root <none> and password file <none>
I20260925 02:27:14.467120 14557 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:14.467202 14557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:14.467334 14557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:14.468238 14557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "e771fe07ccba49d18ffa9657942f20ef"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "8f4bf33bf26d4b8c3114ac8f9a20eb22"
server_key_iv: "d424ab84c6a3c91ecd97455e8b4bb43c"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.468588 14557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "e771fe07ccba49d18ffa9657942f20ef"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "8f4bf33bf26d4b8c3114ac8f9a20eb22"
server_key_iv: "d424ab84c6a3c91ecd97455e8b4bb43c"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.469739 14557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:14.470331 14572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:14.470484 14557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:14.470566 14557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "e771fe07ccba49d18ffa9657942f20ef"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "8f4bf33bf26d4b8c3114ac8f9a20eb22"
server_key_iv: "d424ab84c6a3c91ecd97455e8b4bb43c"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.470633 14557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:14.477406 14557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:14.479233 14557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:14.479367 14557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:14.482816 14557 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:33607
I20260925 02:27:14.482844 14627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:33607 every 8 connection(s)
I20260925 02:27:14.483163 14557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:14.483876 14628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:14.485773 14628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef: Bootstrap starting.
I20260925 02:27:14.486270 14628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:14.486508 14628 log.cc:824] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:14.487119 14628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef: No bootstrap required, opened a new log
I20260925 02:27:14.488207 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 14557
I20260925 02:27:14.488314 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:14.488534 12147 external_mini_cluster.cc:1428] Setting key a561d911d84761a61b3e86a5b00ac108
I20260925 02:27:14.488796 14628 raft_consensus.cc:348] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e771fe07ccba49d18ffa9657942f20ef" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33607 } }
I20260925 02:27:14.488960 14628 raft_consensus.cc:374] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:14.489018 14628 raft_consensus.cc:729] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e771fe07ccba49d18ffa9657942f20ef, State: Initialized, Role: FOLLOWER
I20260925 02:27:14.489166 14628 consensus_queue.cc:260] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e771fe07ccba49d18ffa9657942f20ef" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33607 } }
I20260925 02:27:14.489254 14628 raft_consensus.cc:388] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:14.489284 14628 raft_consensus.cc:482] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:14.489328 14628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:14.489980 14628 raft_consensus.cc:504] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e771fe07ccba49d18ffa9657942f20ef" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33607 } }
I20260925 02:27:14.490088 14628 leader_election.cc:302] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e771fe07ccba49d18ffa9657942f20ef; no voters:
I20260925 02:27:14.490257 14628 leader_election.cc:288] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:14.490410 14634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:14.490505 14628 sys_catalog.cc:564] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:14.490540 14634 raft_consensus.cc:686] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [term 1 LEADER]: Becoming Leader. State: Replica: e771fe07ccba49d18ffa9657942f20ef, State: Running, Role: LEADER
I20260925 02:27:14.490617 14634 consensus_queue.cc:237] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e771fe07ccba49d18ffa9657942f20ef" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33607 } }
I20260925 02:27:14.490933 14634 sys_catalog.cc:455] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader e771fe07ccba49d18ffa9657942f20ef. Latest consensus state: current_term: 1 leader_uuid: "e771fe07ccba49d18ffa9657942f20ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e771fe07ccba49d18ffa9657942f20ef" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33607 } } }
I20260925 02:27:14.491007 14634 sys_catalog.cc:458] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:14.490947 14635 sys_catalog.cc:455] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e771fe07ccba49d18ffa9657942f20ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e771fe07ccba49d18ffa9657942f20ef" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33607 } } }
I20260925 02:27:14.492770 14635 sys_catalog.cc:458] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef [sys.catalog]: This master's current role is: LEADER
W20260925 02:27:14.492277 14648 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:27:14.492825 14648 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:27:14.492882 14640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:14.493017 14640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:14.494568 14640 catalog_manager.cc:1347] Generated new cluster ID: d09dbea7674648a98ad94b4eea131341
I20260925 02:27:14.494611 14640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:27:14 dist-test-slave-dn4n krb5kdc[14544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:14.499805 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:14.489013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41554 (local address 127.11.220.254:33607)
0925 02:27:14.489201 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:14.489214 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:14.489237 (+ 23us) server_negotiation.cc:407] Connection header received
0925 02:27:14.489380 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:14.489387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:14.489435 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:14.489542 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:14.490289 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.491497 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.492658 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.493249 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.493357 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:14.495769 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:14.495773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:14.495773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:14.495783 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:14.497572 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:14.497939 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:14.497942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:14.497942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:14.498043 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:14.498237 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:14.498238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:14.498239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:14.498495 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:14.498990 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:14.499490 (+ 500us) server_negotiation.cc:299] Negotiation successful
0925 02:27:14.499626 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260925 02:27:14.511090 14640 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:14.511535 14640 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:14.515439 14640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e771fe07ccba49d18ffa9657942f20ef: Generated new TSK 0
I20260925 02:27:14.515599 14640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:14.540933 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33607
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:14.613844 14654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:14.614068 14654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:14.614091 14654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:14.615080 14654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:14.615110 14654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:14.615134 14654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:14.616483 14654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:14.616648 14654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:14.616808 14654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:14.621392 14659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:14.621367 14662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:14.621367 14660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:14.621580 14654 server_base.cc:1034] running on GCE node
I20260925 02:27:14.621799 14654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:14.622018 14654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:14.623152 14654 hybrid_clock.cc:648] HybridClock initialized: now 1790303234623143 us; error 35 us; skew 500 ppm
Sep 25 02:27:14 dist-test-slave-dn4n krb5kdc[14544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:14.625648 14654 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:14.626085 14654 webserver.cc:460] Webserver started at http://127.11.220.193:41151/ using document root <none> and password file <none>
I20260925 02:27:14.626310 14654 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:14.626403 14654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:14.626545 14654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:14.627473 14654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "4e512c3fdc3c4879b1e1dbccbc019067"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "8db663901ba3f8f5ac83aefc3be19f54"
server_key_iv: "b09cc8fa474e0b5a74cdb8e8a995a808"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.627815 14654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "4e512c3fdc3c4879b1e1dbccbc019067"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "8db663901ba3f8f5ac83aefc3be19f54"
server_key_iv: "b09cc8fa474e0b5a74cdb8e8a995a808"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.629052 14654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:14.629653 14669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:14.629801 14654 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:14.629889 14654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "4e512c3fdc3c4879b1e1dbccbc019067"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "8db663901ba3f8f5ac83aefc3be19f54"
server_key_iv: "b09cc8fa474e0b5a74cdb8e8a995a808"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.629966 14654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:14.647879 14654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:14.649847 14654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:14.649981 14654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:14.650189 14654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:14.650445 14654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:14.650491 14654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:14.650521 14654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:14.650545 14654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:14.655689 14654 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:34505
I20260925 02:27:14.655725 14783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:34505 every 8 connection(s)
I20260925 02:27:14.656011 14654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
Sep 25 02:27:14 dist-test-slave-dn4n krb5kdc[14544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:14.663607 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:14.656650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:36319 (local address 127.11.220.254:33607)
0925 02:27:14.656710 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:14.656712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:14.656867 (+ 155us) server_negotiation.cc:407] Connection header received
0925 02:27:14.657293 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:14.657295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:14.657306 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:14.657345 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:14.658110 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.658543 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.659415 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.659596 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.659656 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:14.661393 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:14.661395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:14.661395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:14.661401 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:14.662388 (+ 987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:14.662755 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:14.662757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:14.662757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:14.662812 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:14.663028 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:14.663029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:14.663029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:14.663143 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:14.663208 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:14.663451 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 02:27:14.663526 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 02:27:14.663753 14786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:14.656634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33607 (local address 127.11.220.193:36319)
0925 02:27:14.656771 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:14.656798 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:14.657188 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:14.657415 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:14.657418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:14.657862 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:14.658012 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:14.658016 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.658632 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.658633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:14.659333 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:14.659335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.659677 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.659678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:14.659778 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:14.660148 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:14.660155 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:14.661308 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:14.662480 (+ 1172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:14.662487 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:14.662489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:14.662684 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:14.662878 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:14.662878 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:14.662879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:14.662969 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:14.663214 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:14.663221 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:14.663333 (+ 112us) client_negotiation.cc:769] Sending connection context
0925 02:27:14.663419 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 02:27:14.663550 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260925 02:27:14.664194 14784 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33607
I20260925 02:27:14.664261 14784 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:14.664424 14784 heartbeater.cc:511] Master 127.11.220.254:33607 requested a full tablet report, sending...
I20260925 02:27:14.664646 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 14654
I20260925 02:27:14.664741 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:14.664896 12147 external_mini_cluster.cc:1428] Setting key a79c49ba3189d2df86a984d611cbb57e
I20260925 02:27:14.664980 14590 ts_manager.cc:194] Registered new tserver with Master: 4e512c3fdc3c4879b1e1dbccbc019067 (127.11.220.193:34505)
I20260925 02:27:14.665568 14590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:36319
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:14.686877 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33607
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:14.760995 14789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:14.761245 14789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:14.761269 14789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:14.762529 14789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:14.762569 14789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:14.762595 14789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:14.764072 14789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:14.764308 14789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:14.764573 14789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:14.769917 14789 server_base.cc:1034] running on GCE node
W20260925 02:27:14.769822 14794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:14.769815 14797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:14.769811 14795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:14.770399 14789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:14.770615 14789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:14.771756 14789 hybrid_clock.cc:648] HybridClock initialized: now 1790303234771744 us; error 27 us; skew 500 ppm
Sep 25 02:27:14 dist-test-slave-dn4n krb5kdc[14544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:14.774328 14789 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:14.774719 14789 webserver.cc:460] Webserver started at http://127.11.220.194:37651/ using document root <none> and password file <none>
I20260925 02:27:14.774956 14789 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:14.775029 14789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:14.775162 14789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:14.776118 14789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "151866b16a784b34b528d32f1acf0115"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "5a263dce8339713317913799667a8d45"
server_key_iv: "c59d8ca4e1f155a5c84220870a858b4a"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.776526 14789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "151866b16a784b34b528d32f1acf0115"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "5a263dce8339713317913799667a8d45"
server_key_iv: "c59d8ca4e1f155a5c84220870a858b4a"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.777781 14789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:14.778468 14804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:14.778648 14789 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:14.778744 14789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "151866b16a784b34b528d32f1acf0115"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "5a263dce8339713317913799667a8d45"
server_key_iv: "c59d8ca4e1f155a5c84220870a858b4a"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.778829 14789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:14.791450 14789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:14.793308 14789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:14.793408 14789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:14.793558 14789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:14.793833 14789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:14.793864 14789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:14.793884 14789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:14.793931 14789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:14.799386 14789 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:44253
I20260925 02:27:14.799453 14919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:44253 every 8 connection(s)
I20260925 02:27:14.799755 14789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:14.800632 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 14789
I20260925 02:27:14.800746 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:14.800904 12147 external_mini_cluster.cc:1428] Setting key 700c17e4a9135b193dbb1db34c50a76f
Sep 25 02:27:14 dist-test-slave-dn4n krb5kdc[14544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
I20260925 02:27:14.809487 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:14.800427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:42921 (local address 127.11.220.254:33607)
0925 02:27:14.800528 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:14.800531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:14.800705 (+ 174us) server_negotiation.cc:407] Connection header received
0925 02:27:14.801324 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:14.801325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:14.801335 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:14.801381 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:14.802325 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.803045 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.804522 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.804716 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.804767 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:14.806767 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:14.806769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:14.806769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:14.806777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:14.808113 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:14.808532 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:14.808534 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:14.808534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:14.808593 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:14.808849 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:14.808850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:14.808850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:14.808955 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:14.809029 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:14.809311 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 02:27:14.809399 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 02:27:14.809625 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:14.800399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33607 (local address 127.11.220.194:42921)
0925 02:27:14.800570 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:14.800606 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:14.801228 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:14.801451 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:14.801455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:14.802053 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:14.802223 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:14.802227 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.803119 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.803120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:14.804412 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:14.804415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.804799 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.804800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:14.804952 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:14.805441 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:14.805449 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:14.806675 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:14.808199 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:14.808207 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:14.808209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:14.808417 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:14.808660 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:14.808661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:14.808661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:14.808775 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:14.809021 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:14.809028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:14.809157 (+ 129us) client_negotiation.cc:769] Sending connection context
0925 02:27:14.809276 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 02:27:14.809419 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260925 02:27:14.810091 14920 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33607
I20260925 02:27:14.810189 14920 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:14.810343 14920 heartbeater.cc:511] Master 127.11.220.254:33607 requested a full tablet report, sending...
I20260925 02:27:14.810681 14590 ts_manager.cc:194] Registered new tserver with Master: 151866b16a784b34b528d32f1acf0115 (127.11.220.194:44253)
I20260925 02:27:14.811060 14590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:42921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:14.824249 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33607
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:14.898653 14925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:14.898808 14925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:14.898828 14925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:14.899820 14925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:14.899848 14925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:14.899869 14925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:14.901137 14925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:14.901288 14925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:14.901443 14925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:14.906131 14930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:14.906147 14931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:14.906147 14933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:14.906306 14925 server_base.cc:1034] running on GCE node
I20260925 02:27:14.906486 14925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:14.906684 14925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:14.907829 14925 hybrid_clock.cc:648] HybridClock initialized: now 1790303234907811 us; error 43 us; skew 500 ppm
Sep 25 02:27:14 dist-test-slave-dn4n krb5kdc[14544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:14.909986 14925 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:14.910420 14925 webserver.cc:460] Webserver started at http://127.11.220.195:40089/ using document root <none> and password file <none>
I20260925 02:27:14.910624 14925 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:14.910684 14925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:14.910826 14925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:14.911762 14925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "75f1c2a2f795485189c47e0e8bda7ebe"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "accf93910baa36ca10887be5d1131a57"
server_key_iv: "b014bc9e55232e0ca556308e43115d81"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.912096 14925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "75f1c2a2f795485189c47e0e8bda7ebe"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "accf93910baa36ca10887be5d1131a57"
server_key_iv: "b014bc9e55232e0ca556308e43115d81"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.913385 14925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:27:14.914022 14941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:14.914165 14925 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:14.914245 14925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "75f1c2a2f795485189c47e0e8bda7ebe"
format_stamp: "Formatted at 2026-09-25 02:27:14 on dist-test-slave-dn4n"
server_key: "accf93910baa36ca10887be5d1131a57"
server_key_iv: "b014bc9e55232e0ca556308e43115d81"
server_key_version: "encryptionkey@0"
I20260925 02:27:14.914312 14925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:14.944096 14925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:14.946107 14925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:14.946249 14925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:14.946439 14925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:14.946697 14925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:14.946745 14925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:14.946779 14925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:14.946803 14925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:14.952072 14925 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:39133
I20260925 02:27:14.952118 15056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:39133 every 8 connection(s)
I20260925 02:27:14.952385 14925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:27:14 dist-test-slave-dn4n krb5kdc[14544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303234, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:14.958401 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 14925
I20260925 02:27:14.958499 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:14.958724 12147 external_mini_cluster.cc:1428] Setting key 86e5b9bb21801ce03aa251cffb39307d
I20260925 02:27:14.960621 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:14.953037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:58915 (local address 127.11.220.254:33607)
0925 02:27:14.953111 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:14.953113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:14.953293 (+ 180us) server_negotiation.cc:407] Connection header received
0925 02:27:14.953721 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:14.953722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:14.953732 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:14.953778 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:14.954505 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.955001 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.955908 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.956108 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.956154 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:14.957898 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:14.957899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:14.957899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:14.957905 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:14.959434 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:14.959845 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:14.959846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:14.959846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:14.959896 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:14.960105 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:14.960106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:14.960107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:14.960199 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:14.960246 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:14.960491 (+ 245us) server_negotiation.cc:299] Negotiation successful
0925 02:27:14.960558 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:27:14.960768 15059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:14.952994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33607 (local address 127.11.220.195:58915)
0925 02:27:14.953158 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:14.953190 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:14.953638 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:14.953820 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:14.953823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:14.954281 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:14.954438 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:14.954442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.955081 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.955082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:14.955835 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:14.955837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:14.956174 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:14.956175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:14.956260 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:14.956659 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:14.956675 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:14.957822 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:14.959509 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:14.959516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:14.959518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:14.959782 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:14.959951 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:14.959952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:14.959952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:14.960048 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:14.960250 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:14.960254 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:14.960351 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 02:27:14.960425 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 02:27:14.960592 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260925 02:27:14.961148 15057 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33607
I20260925 02:27:14.961232 15057 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:14.961400 15057 heartbeater.cc:511] Master 127.11.220.254:33607 requested a full tablet report, sending...
I20260925 02:27:14.961752 14590 ts_manager.cc:194] Registered new tserver with Master: 75f1c2a2f795485189c47e0e8bda7ebe (127.11.220.195:39133)
I20260925 02:27:14.962097 14590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:58915
I20260925 02:27:14.970431 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 02:27:14.984545 14632 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:14.983651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41556 (local address 127.11.220.254:33607)
0925 02:27:14.983723 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:14.983725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:14.983807 (+ 82us) server_negotiation.cc:407] Connection header received
0925 02:27:14.983902 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:14.983903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:14.983912 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:14.983983 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:14.984449 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41556: BlockingRecv error: recv got EOF from 127.0.0.1:41556 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:27:14.985363 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 14654
I20260925 02:27:14.990193 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 14789
I20260925 02:27:14.994210 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 14925
I20260925 02:27:14.998128 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 14557
2026-09-25T02:27:14Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1652 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:15 dist-test-slave-dn4n krb5kdc[15069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:15 dist-test-slave-dn4n krb5kdc[15069](info): set up 2 sockets
Sep 25 02:27:15 dist-test-slave-dn4n krb5kdc[15069](info): commencing operation
krb5kdc: starting...
W20260925 02:27:15.946022 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:16 dist-test-slave-dn4n krb5kdc[15069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:16.045224 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:35549
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:35549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:16.115417 15082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:16.115617 15082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:16.115649 15082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:16.116768 15082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:16.116832 15082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:16.116855 15082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:16.116878 15082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:16.116905 15082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:16.118270 15082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:35549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:35549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:16.118507 15082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:16.118697 15082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:16.123221 15088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:16.123215 15087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:16.123350 15082 server_base.cc:1034] running on GCE node
W20260925 02:27:16.123214 15090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:16.123607 15082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:16.123900 15082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:16.125052 15082 hybrid_clock.cc:648] HybridClock initialized: now 1790303236125050 us; error 41 us; skew 500 ppm
Sep 25 02:27:16 dist-test-slave-dn4n krb5kdc[15069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303236, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:16.127151 15082 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:16.127514 15082 webserver.cc:460] Webserver started at http://127.11.220.254:37517/ using document root <none> and password file <none>
I20260925 02:27:16.127728 15082 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:16.127789 15082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:16.127923 15082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:16.128969 15082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "470796ab62eb406b8f3af307564e71df"
format_stamp: "Formatted at 2026-09-25 02:27:16 on dist-test-slave-dn4n"
server_key: "9994db0296ef10c67d674e8e7d5f6234"
server_key_iv: "49106c565d1c264c6005f2cd6776272a"
server_key_version: "encryptionkey@0"
I20260925 02:27:16.129292 15082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "470796ab62eb406b8f3af307564e71df"
format_stamp: "Formatted at 2026-09-25 02:27:16 on dist-test-slave-dn4n"
server_key: "9994db0296ef10c67d674e8e7d5f6234"
server_key_iv: "49106c565d1c264c6005f2cd6776272a"
server_key_version: "encryptionkey@0"
I20260925 02:27:16.130465 15082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:16.131062 15097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:16.131225 15082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:16.131309 15082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "470796ab62eb406b8f3af307564e71df"
format_stamp: "Formatted at 2026-09-25 02:27:16 on dist-test-slave-dn4n"
server_key: "9994db0296ef10c67d674e8e7d5f6234"
server_key_iv: "49106c565d1c264c6005f2cd6776272a"
server_key_version: "encryptionkey@0"
I20260925 02:27:16.131385 15082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:16.146482 15082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:16.147938 15082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:16.148083 15082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:16.151695 15082 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:35549
I20260925 02:27:16.151721 15149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:35549 every 8 connection(s)
I20260925 02:27:16.152120 15082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:16.152751 15150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:16.154696 15150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df: Bootstrap starting.
I20260925 02:27:16.155280 15150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:16.155541 15150 log.cc:824] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:16.156144 15150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df: No bootstrap required, opened a new log
I20260925 02:27:16.157545 15150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470796ab62eb406b8f3af307564e71df" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35549 } }
I20260925 02:27:16.157660 15150 raft_consensus.cc:374] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:16.157723 15150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470796ab62eb406b8f3af307564e71df, State: Initialized, Role: FOLLOWER
I20260925 02:27:16.157850 15150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470796ab62eb406b8f3af307564e71df" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35549 } }
I20260925 02:27:16.157928 15150 raft_consensus.cc:388] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:16.157968 15150 raft_consensus.cc:482] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:16.158011 15150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:16.158676 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 15082
I20260925 02:27:16.158737 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:16.158788 15150 raft_consensus.cc:504] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470796ab62eb406b8f3af307564e71df" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35549 } }
I20260925 02:27:16.158874 12147 external_mini_cluster.cc:1428] Setting key b3bef128bcc53aec574d64a45775481e
I20260925 02:27:16.158897 15150 leader_election.cc:302] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 470796ab62eb406b8f3af307564e71df; no voters:
I20260925 02:27:16.159054 15150 leader_election.cc:288] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:16.159366 15150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:16.159087 15154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:16.160524 15154 raft_consensus.cc:686] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [term 1 LEADER]: Becoming Leader. State: Replica: 470796ab62eb406b8f3af307564e71df, State: Running, Role: LEADER
I20260925 02:27:16.160609 15154 consensus_queue.cc:237] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470796ab62eb406b8f3af307564e71df" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35549 } }
I20260925 02:27:16.161078 15155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "470796ab62eb406b8f3af307564e71df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470796ab62eb406b8f3af307564e71df" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35549 } } }
I20260925 02:27:16.161149 15155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:16.161353 15168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 470796ab62eb406b8f3af307564e71df. Latest consensus state: current_term: 1 leader_uuid: "470796ab62eb406b8f3af307564e71df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470796ab62eb406b8f3af307564e71df" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35549 } } }
I20260925 02:27:16.161448 15168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df [sys.catalog]: This master's current role is: LEADER
W20260925 02:27:16.161829 15170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:27:16.161900 15170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:27:16.162809 15171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:16.162981 15171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:16.164388 15171 catalog_manager.cc:1347] Generated new cluster ID: 8f3b747c094b411595700c73566e1a98
I20260925 02:27:16.164453 15171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:27:16 dist-test-slave-dn4n krb5kdc[15069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:16.170361 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:16.159219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.11.220.254:35549)
0925 02:27:16.160890 (+ 1671us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:16.160898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:16.160929 (+ 31us) server_negotiation.cc:407] Connection header received
0925 02:27:16.161047 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:16.161054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:16.161113 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:16.161278 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:16.161885 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:16.162695 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:16.163659 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:16.164203 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:16.164301 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:16.166712 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:16.166716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:16.166728 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:16.166737 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:16.168883 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:16.169255 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:16.169257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:16.169258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:16.169358 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:16.169541 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:16.169542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:16.169542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:16.169834 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:16.169981 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:16.170046 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 02:27:16.170166 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1615,"thread_start_us":55,"threads_started":1}
I20260925 02:27:16.171154 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:27:16.188884 15171 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:16.189016 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:16.185210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.11.220.254:35549)
0925 02:27:16.185268 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:16.185270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:16.185361 (+ 91us) server_negotiation.cc:407] Connection header received
0925 02:27:16.185413 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:16.185414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:16.185423 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:16.185472 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:16.185792 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:16.186424 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:16.187466 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:16.187782 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:16.187845 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:16.188565 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:16.188567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:16.188567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:16.188789 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:16.188804 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:16.188811 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:16.188862 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:16.188926 (+ 64us) server_negotiation.cc:299] Negotiation successful
0925 02:27:16.188948 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:27:16.189559 15171 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:16.195351 15171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 470796ab62eb406b8f3af307564e71df: Generated new TSK 0
I20260925 02:27:16.195514 15171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:27:16.204632 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 15082
2026-09-25T02:27:16Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1204 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:16 dist-test-slave-dn4n krb5kdc[15185](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:16 dist-test-slave-dn4n krb5kdc[15185](info): set up 2 sockets
Sep 25 02:27:16 dist-test-slave-dn4n krb5kdc[15185](info): commencing operation
krb5kdc: starting...
W20260925 02:27:17.140413 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:17.240068 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:38387
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:37549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:38387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:17.314250 15198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:17.314406 15198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:17.314424 15198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:17.315483 15198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:17.315521 15198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:17.315534 15198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:17.315546 15198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:17.315594 15198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:17.316982 15198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:37549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:38387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:38387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:17.317216 15198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:17.317411 15198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:17.321904 15206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:17.321897 15204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:17.321931 15198 server_base.cc:1034] running on GCE node
W20260925 02:27:17.321911 15203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:17.322359 15198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:17.322636 15198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:17.323789 15198 hybrid_clock.cc:648] HybridClock initialized: now 1790303237323778 us; error 44 us; skew 500 ppm
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:17.329949 15198 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:17.330320 15198 webserver.cc:460] Webserver started at http://127.11.220.254:33095/ using document root <none> and password file <none>
I20260925 02:27:17.330523 15198 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:17.330583 15198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:17.330726 15198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:17.331629 15198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "d75a12332c1740aaabfcd4aa19d7432f"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "b337eceddf18bf2ca349178afe8c4b96"
server_key_iv: "4afbe0199b5039df21127e4550c830a4"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.331950 15198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "d75a12332c1740aaabfcd4aa19d7432f"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "b337eceddf18bf2ca349178afe8c4b96"
server_key_iv: "4afbe0199b5039df21127e4550c830a4"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.333218 15198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:27:17.333899 15213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:17.334051 15198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:17.334139 15198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "d75a12332c1740aaabfcd4aa19d7432f"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "b337eceddf18bf2ca349178afe8c4b96"
server_key_iv: "4afbe0199b5039df21127e4550c830a4"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.334213 15198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:17.345897 15198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:17.347851 15198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:17.348001 15198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:17.351707 15198 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:38387
I20260925 02:27:17.351737 15266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:38387 every 8 connection(s)
I20260925 02:27:17.352038 15198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:17.353066 15267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:17.353559 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 15198
I20260925 02:27:17.353662 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:17.353820 12147 external_mini_cluster.cc:1428] Setting key 991dc6c7f532950689633da0d4a661bc
I20260925 02:27:17.355959 15267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f: Bootstrap starting.
I20260925 02:27:17.356554 15267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:17.356897 15267 log.cc:824] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:17.357609 15267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f: No bootstrap required, opened a new log
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:17.359946 15267 raft_consensus.cc:348] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75a12332c1740aaabfcd4aa19d7432f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38387 } }
I20260925 02:27:17.360105 15267 raft_consensus.cc:374] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:17.360152 15267 raft_consensus.cc:729] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d75a12332c1740aaabfcd4aa19d7432f, State: Initialized, Role: FOLLOWER
I20260925 02:27:17.360267 15267 consensus_queue.cc:260] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75a12332c1740aaabfcd4aa19d7432f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38387 } }
I20260925 02:27:17.360361 15267 raft_consensus.cc:388] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:17.360405 15267 raft_consensus.cc:482] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:17.360483 15267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:17.361136 15267 raft_consensus.cc:504] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75a12332c1740aaabfcd4aa19d7432f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38387 } }
I20260925 02:27:17.361249 15267 leader_election.cc:302] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d75a12332c1740aaabfcd4aa19d7432f; no voters:
I20260925 02:27:17.361390 15267 leader_election.cc:288] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:17.361440 15272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:17.361599 15272 raft_consensus.cc:686] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [term 1 LEADER]: Becoming Leader. State: Replica: d75a12332c1740aaabfcd4aa19d7432f, State: Running, Role: LEADER
I20260925 02:27:17.361609 15267 sys_catalog.cc:564] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:17.361711 15272 consensus_queue.cc:237] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75a12332c1740aaabfcd4aa19d7432f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38387 } }
I20260925 02:27:17.362154 15273 sys_catalog.cc:455] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d75a12332c1740aaabfcd4aa19d7432f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75a12332c1740aaabfcd4aa19d7432f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38387 } } }
I20260925 02:27:17.362340 15273 sys_catalog.cc:458] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:17.362286 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d75a12332c1740aaabfcd4aa19d7432f. Latest consensus state: current_term: 1 leader_uuid: "d75a12332c1740aaabfcd4aa19d7432f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d75a12332c1740aaabfcd4aa19d7432f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38387 } } }
I20260925 02:27:17.362666 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:17.362856 15281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:17.363353 15281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:17.364010 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:17.354306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.11.220.254:38387)
0925 02:27:17.354460 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:17.354468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:17.354501 (+ 33us) server_negotiation.cc:407] Connection header received
0925 02:27:17.354883 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:17.354889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:17.354929 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:17.355036 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:17.355754 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.356642 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.357587 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.357989 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.358076 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:17.360380 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:17.360384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:17.360384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:17.360393 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:17.362484 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:17.362881 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:17.362884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:17.362885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:17.362984 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:17.363202 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:17.363203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:17.363204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:17.363470 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:17.363619 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:17.363722 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 02:27:17.363845 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260925 02:27:17.364934 15281 catalog_manager.cc:1347] Generated new cluster ID: cf0e6787083a434cbcd9ae09c9b0eaf9
I20260925 02:27:17.364974 15281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:17.386161 15281 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:17.386656 15281 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:17.398687 15281 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d75a12332c1740aaabfcd4aa19d7432f: Generated new TSK 0
I20260925 02:27:17.398840 15281 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:17.429762 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:38387
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:37549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:17.516667 15294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:17.516866 15294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:17.516898 15294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:17.517928 15294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:17.517985 15294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:17.518028 15294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:17.519483 15294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:37549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:38387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:17.519723 15294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:17.519917 15294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:17.524698 15300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:17.524708 15294 server_base.cc:1034] running on GCE node
W20260925 02:27:17.524682 15301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:17.524682 15303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:17.525100 15294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:17.525348 15294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:17.526481 15294 hybrid_clock.cc:648] HybridClock initialized: now 1790303237526472 us; error 36 us; skew 500 ppm
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:17.528775 15294 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:17.529178 15294 webserver.cc:460] Webserver started at http://127.11.220.193:36513/ using document root <none> and password file <none>
I20260925 02:27:17.529372 15294 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:17.529431 15294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:17.529578 15294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:17.530457 15294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "59e02d38f3f84408952e93c843666ea2"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "700c18c66ceed84915542741abf5d9ac"
server_key_iv: "cc9fde1959ac22b78ab556ca38995375"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.530784 15294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "59e02d38f3f84408952e93c843666ea2"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "700c18c66ceed84915542741abf5d9ac"
server_key_iv: "cc9fde1959ac22b78ab556ca38995375"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.531949 15294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 02:27:17.532621 15310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:17.532812 15294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:17.532905 15294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "59e02d38f3f84408952e93c843666ea2"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "700c18c66ceed84915542741abf5d9ac"
server_key_iv: "cc9fde1959ac22b78ab556ca38995375"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.532987 15294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:17.548753 15294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:17.550750 15294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:17.550889 15294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:17.551111 15294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:17.551381 15294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:17.551431 15294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:17.551467 15294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:17.551493 15294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:17.556921 15294 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:46067
I20260925 02:27:17.556969 15423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:46067 every 8 connection(s)
I20260925 02:27:17.557238 15294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:17.564069 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 15294
I20260925 02:27:17.564167 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:17.564358 12147 external_mini_cluster.cc:1428] Setting key 5a2632ec46c4f2633f7e0d6b81dff386
I20260925 02:27:17.565886 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:17.557898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:33299 (local address 127.11.220.254:38387)
0925 02:27:17.557938 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:17.557940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:17.558111 (+ 171us) server_negotiation.cc:407] Connection header received
0925 02:27:17.558529 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:17.558530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:17.558549 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:17.558601 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:17.559391 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.559840 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.560767 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.560961 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.561012 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:17.562883 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:17.562884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:17.562884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:17.562889 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:17.564518 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:17.564995 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:17.564996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:17.564996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:17.565056 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:17.565327 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:17.565328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:17.565328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:17.565421 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:17.565468 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:17.565769 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 02:27:17.565827 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:27:17.566124 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:17.557863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:38387 (local address 127.11.220.193:33299)
0925 02:27:17.558015 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:17.558052 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:17.558449 (+ 397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:17.558682 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:17.558685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:17.559165 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:17.559322 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:17.559325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.559909 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.559910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:17.560699 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:17.560702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.561026 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.561027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:17.561115 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:17.561477 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:17.561501 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:17.562810 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:17.564596 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:17.564603 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:17.564605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:17.564874 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:17.565128 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:17.565128 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:17.565129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:17.565264 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:17.565483 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:17.565489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:17.565648 (+ 159us) client_negotiation.cc:769] Sending connection context
0925 02:27:17.565748 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 02:27:17.565898 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260925 02:27:17.566650 15424 heartbeater.cc:347] Connected to a master server at 127.11.220.254:38387
I20260925 02:27:17.566720 15424 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:17.566884 15424 heartbeater.cc:511] Master 127.11.220.254:38387 requested a full tablet report, sending...
I20260925 02:27:17.567413 15230 ts_manager.cc:194] Registered new tserver with Master: 59e02d38f3f84408952e93c843666ea2 (127.11.220.193:46067)
I20260925 02:27:17.568238 15230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:33299
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:17.586486 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:38387
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:37549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:17.658036 15429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:17.658209 15429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:17.658231 15429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:17.659183 15429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:17.659211 15429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:17.659235 15429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:17.660598 15429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:37549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:38387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:17.660765 15429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:17.660903 15429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:17.665464 15429 server_base.cc:1034] running on GCE node
W20260925 02:27:17.665393 15434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:17.665418 15435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:17.665395 15437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:17.665818 15429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:17.666055 15429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:17.667196 15429 hybrid_clock.cc:648] HybridClock initialized: now 1790303237667183 us; error 27 us; skew 500 ppm
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:17.669308 15429 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:17.669660 15429 webserver.cc:460] Webserver started at http://127.11.220.194:41515/ using document root <none> and password file <none>
I20260925 02:27:17.669852 15429 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:17.669912 15429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:17.670081 15429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:17.670969 15429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "d70492dae7cc4542aec6c4e37887c7de"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "ac57985a4f3412f30c193fc8cdd48870"
server_key_iv: "1a5a0e568c5b3f7b4aeadcb03d0e6469"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.671325 15429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "d70492dae7cc4542aec6c4e37887c7de"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "ac57985a4f3412f30c193fc8cdd48870"
server_key_iv: "1a5a0e568c5b3f7b4aeadcb03d0e6469"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.672518 15429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:17.673224 15445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:17.673390 15429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:17.673487 15429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "d70492dae7cc4542aec6c4e37887c7de"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "ac57985a4f3412f30c193fc8cdd48870"
server_key_iv: "1a5a0e568c5b3f7b4aeadcb03d0e6469"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.673563 15429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:17.688148 15429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:17.689699 15429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:17.689841 15429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:17.690030 15429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:17.690294 15429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:17.690341 15429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:17.690376 15429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:17.690400 15429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:17.695421 15429 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:39741
I20260925 02:27:17.695468 15558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:39741 every 8 connection(s)
I20260925 02:27:17.695725 15429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:17.700296 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 15429
I20260925 02:27:17.700417 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:17.700620 12147 external_mini_cluster.cc:1428] Setting key 867db270651e38d9263315e2e7fea25a
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:17.704418 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:17.696331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:38387 (local address 127.11.220.194:57967)
0925 02:27:17.696507 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:17.696568 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:17.697011 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:17.697198 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:17.697201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:17.697730 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:17.697913 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:17.697917 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.698540 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.698540 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:17.699301 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:17.699303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.699631 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.699632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:17.699718 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:17.700144 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:17.700151 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:17.701622 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:17.703144 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:17.703151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:17.703153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:17.703442 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:17.703625 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:17.703625 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:17.703626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:17.703741 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:17.704013 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:17.704018 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:17.704107 (+ 89us) client_negotiation.cc:769] Sending connection context
0925 02:27:17.704175 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 02:27:17.704269 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260925 02:27:17.704435 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:17.696388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:57967 (local address 127.11.220.254:38387)
0925 02:27:17.696493 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:17.696495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:17.696662 (+ 167us) server_negotiation.cc:407] Connection header received
0925 02:27:17.697088 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:17.697089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:17.697099 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:17.697149 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:17.697987 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.698460 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.699371 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.699544 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.699591 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:17.701718 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:17.701720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:17.701720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:17.701728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:17.703067 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:17.703516 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:17.703516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:17.703517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:17.703562 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:17.703807 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:17.703808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:17.703808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:17.703942 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:17.704019 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:17.704216 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 02:27:17.704292 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 02:27:17.704955 15559 heartbeater.cc:347] Connected to a master server at 127.11.220.254:38387
I20260925 02:27:17.705049 15559 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:17.705216 15559 heartbeater.cc:511] Master 127.11.220.254:38387 requested a full tablet report, sending...
I20260925 02:27:17.705597 15230 ts_manager.cc:194] Registered new tserver with Master: d70492dae7cc4542aec6c4e37887c7de (127.11.220.194:39741)
I20260925 02:27:17.705962 15230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:57967
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:17.723311 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:38387
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:37549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:17.796234 15564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:17.796394 15564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:17.796415 15564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:17.797459 15564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:17.797513 15564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:17.797539 15564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:17.799026 15564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:37549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:38387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:17.799253 15564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:17.799466 15564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:17.803956 15569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:17.803967 15572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:17.803990 15570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:17.804234 15564 server_base.cc:1034] running on GCE node
I20260925 02:27:17.804425 15564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:17.804690 15564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:17.805848 15564 hybrid_clock.cc:648] HybridClock initialized: now 1790303237805838 us; error 27 us; skew 500 ppm
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:17.808116 15564 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:17.808497 15564 webserver.cc:460] Webserver started at http://127.11.220.195:43219/ using document root <none> and password file <none>
I20260925 02:27:17.808681 15564 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:17.808717 15564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:17.808794 15564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:17.809604 15564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "c0b702e031474c75bb418a3350ab9f16"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "2c2e91734ece7c21f4effe22ea42572f"
server_key_iv: "f5589a75daf0f7918b7c2027f9a7f68f"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.809861 15564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "c0b702e031474c75bb418a3350ab9f16"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "2c2e91734ece7c21f4effe22ea42572f"
server_key_iv: "f5589a75daf0f7918b7c2027f9a7f68f"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.811096 15564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:17.811728 15579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:17.811913 15564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:17.811976 15564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "c0b702e031474c75bb418a3350ab9f16"
format_stamp: "Formatted at 2026-09-25 02:27:17 on dist-test-slave-dn4n"
server_key: "2c2e91734ece7c21f4effe22ea42572f"
server_key_iv: "f5589a75daf0f7918b7c2027f9a7f68f"
server_key_version: "encryptionkey@0"
I20260925 02:27:17.812026 15564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:17.818459 15564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:17.820386 15564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:17.820552 15564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:17.820714 15564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:17.820928 15564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:17.820956 15564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:17.820976 15564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:17.820987 15564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:17.826428 15564 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:40405
I20260925 02:27:17.826484 15694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:40405 every 8 connection(s)
I20260925 02:27:17.826753 15564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:17.827152 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 15564
I20260925 02:27:17.827244 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:17.827402 12147 external_mini_cluster.cc:1428] Setting key 0604bb5964e4560bdec5d408c0687d05
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:17.835322 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:17.827883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:43933 (local address 127.11.220.254:38387)
0925 02:27:17.827924 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:17.827927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:17.828154 (+ 227us) server_negotiation.cc:407] Connection header received
0925 02:27:17.828661 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:17.828662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:17.828686 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:17.828730 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:17.829484 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.829923 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.830772 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.830948 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.830993 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:17.833070 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:17.833071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:17.833072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:17.833077 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:17.834262 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:17.834582 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:17.834583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:17.834583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:17.834626 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:17.834821 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:17.834821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:17.834821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:17.834912 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:17.834957 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:17.835159 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 02:27:17.835257 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:27:17.835459 15697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:17.827873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:38387 (local address 127.11.220.195:43933)
0925 02:27:17.828037 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:17.828072 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:17.828571 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:17.828777 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:17.828780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:17.829264 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:17.829420 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:17.829423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.830004 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.830004 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:17.830710 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:17.830711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.831097 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.831098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:17.831199 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:17.831661 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:17.831668 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:17.832997 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:17.834328 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:17.834334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:17.834335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:17.834522 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:17.834679 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:17.834679 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:17.834680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:17.834771 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:17.834962 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:17.834966 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:17.835059 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 02:27:17.835133 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 02:27:17.835261 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":91,"threads_started":1}
I20260925 02:27:17.835884 15695 heartbeater.cc:347] Connected to a master server at 127.11.220.254:38387
I20260925 02:27:17.835999 15695 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:17.836145 15695 heartbeater.cc:511] Master 127.11.220.254:38387 requested a full tablet report, sending...
I20260925 02:27:17.836535 15230 ts_manager.cc:194] Registered new tserver with Master: c0b702e031474c75bb418a3350ab9f16 (127.11.220.195:40405)
I20260925 02:27:17.836848 15230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:43933
I20260925 02:27:17.839082 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303237, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:17.858341 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:17.852551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.11.220.254:38387)
0925 02:27:17.852606 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:17.852608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:17.852692 (+ 84us) server_negotiation.cc:407] Connection header received
0925 02:27:17.852761 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:17.852762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:17.852771 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:17.852818 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:17.853339 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.853760 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.854432 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:17.854606 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:17.854679 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:17.856587 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:17.856588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:17.856589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:17.856594 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:17.857559 (+ 965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:17.857829 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:17.857830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:17.857830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:17.857871 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:17.858001 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:17.858001 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:17.858001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:17.858105 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:17.858165 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:17.858243 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 02:27:17.858291 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260925 02:27:17.858644 15230 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49206
I20260925 02:27:17.859139 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 15294
I20260925 02:27:17.864114 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 15429
I20260925 02:27:17.867933 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 15564
I20260925 02:27:17.872390 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 15198
2026-09-25T02:27:17Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1669 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15707](info): set up 2 sockets
Sep 25 02:27:17 dist-test-slave-dn4n krb5kdc[15707](info): commencing operation
krb5kdc: starting...
W20260925 02:27:18.805019 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.913s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:18 dist-test-slave-dn4n krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:18.905304 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:33089
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:33089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:18.978708 15720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:18.978897 15720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:18.978929 15720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:18.979935 15720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:18.979992 15720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:18.980013 15720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:18.980055 15720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:18.980083 15720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:18.981417 15720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:33089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:33089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:18.981644 15720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:18.981828 15720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:18.985970 15726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:18.985965 15727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:18.985965 15729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:18.986009 15720 server_base.cc:1034] running on GCE node
I20260925 02:27:18.986325 15720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:18.986523 15720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:18.987653 15720 hybrid_clock.cc:648] HybridClock initialized: now 1790303238987641 us; error 27 us; skew 500 ppm
Sep 25 02:27:18 dist-test-slave-dn4n krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303238, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:18.989838 15720 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:18.990257 15720 webserver.cc:460] Webserver started at http://127.11.220.254:46743/ using document root <none> and password file <none>
I20260925 02:27:18.990442 15720 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:18.990481 15720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:18.990623 15720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:18.991489 15720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "a51063d188a948d09e9db14fe4238ad8"
format_stamp: "Formatted at 2026-09-25 02:27:18 on dist-test-slave-dn4n"
server_key: "ff8aaaa64aaec534bb2286439cb2d199"
server_key_iv: "68b654aaa74987e68f528bbcafd777eb"
server_key_version: "encryptionkey@0"
I20260925 02:27:18.991799 15720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "a51063d188a948d09e9db14fe4238ad8"
format_stamp: "Formatted at 2026-09-25 02:27:18 on dist-test-slave-dn4n"
server_key: "ff8aaaa64aaec534bb2286439cb2d199"
server_key_iv: "68b654aaa74987e68f528bbcafd777eb"
server_key_version: "encryptionkey@0"
I20260925 02:27:18.993017 15720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:18.993722 15736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:18.993894 15720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:18.993986 15720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "a51063d188a948d09e9db14fe4238ad8"
format_stamp: "Formatted at 2026-09-25 02:27:18 on dist-test-slave-dn4n"
server_key: "ff8aaaa64aaec534bb2286439cb2d199"
server_key_iv: "68b654aaa74987e68f528bbcafd777eb"
server_key_version: "encryptionkey@0"
I20260925 02:27:18.994050 15720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:19.003832 15720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:19.005803 15720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:19.005889 15720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:19.009522 15720 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:33089
I20260925 02:27:19.009549 15788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:33089 every 8 connection(s)
I20260925 02:27:19.009948 15720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:19.010454 15789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:19.012267 15789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8: Bootstrap starting.
I20260925 02:27:19.012852 15789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:19.013129 15789 log.cc:824] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:19.013787 15789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8: No bootstrap required, opened a new log
I20260925 02:27:19.015120 15789 raft_consensus.cc:348] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51063d188a948d09e9db14fe4238ad8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33089 } }
I20260925 02:27:19.015234 15789 raft_consensus.cc:374] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:19.015286 15789 raft_consensus.cc:729] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a51063d188a948d09e9db14fe4238ad8, State: Initialized, Role: FOLLOWER
I20260925 02:27:19.015425 15789 consensus_queue.cc:260] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51063d188a948d09e9db14fe4238ad8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33089 } }
I20260925 02:27:19.015492 15789 raft_consensus.cc:388] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:19.015537 15789 raft_consensus.cc:482] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:19.015576 15789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:19.016403 15789 raft_consensus.cc:504] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51063d188a948d09e9db14fe4238ad8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33089 } }
I20260925 02:27:19.016535 15789 leader_election.cc:302] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a51063d188a948d09e9db14fe4238ad8; no voters:
I20260925 02:27:19.016719 15789 leader_election.cc:288] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:19.016757 15792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:19.016935 15792 raft_consensus.cc:686] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [term 1 LEADER]: Becoming Leader. State: Replica: a51063d188a948d09e9db14fe4238ad8, State: Running, Role: LEADER
I20260925 02:27:19.016958 15789 sys_catalog.cc:564] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:19.017057 15792 consensus_queue.cc:237] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51063d188a948d09e9db14fe4238ad8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33089 } }
I20260925 02:27:19.017465 15793 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a51063d188a948d09e9db14fe4238ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51063d188a948d09e9db14fe4238ad8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33089 } } }
I20260925 02:27:19.017479 15795 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a51063d188a948d09e9db14fe4238ad8. Latest consensus state: current_term: 1 leader_uuid: "a51063d188a948d09e9db14fe4238ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51063d188a948d09e9db14fe4238ad8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33089 } } }
I20260925 02:27:19.017638 15793 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:19.017658 15795 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:19.017969 15802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:19.018421 15802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:19.019438 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 15720
I20260925 02:27:19.019546 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:19.019696 12147 external_mini_cluster.cc:1428] Setting key d5a0808c6084ef1e9108ac69b698fbb3
I20260925 02:27:19.020807 15802 catalog_manager.cc:1347] Generated new cluster ID: 3b2b899e2c6b4a14aec0afb0b72b1100
I20260925 02:27:19.020851 15802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[15707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:19.030007 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:19.020088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39512 (local address 127.11.220.254:33089)
0925 02:27:19.020224 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:19.020232 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:19.020262 (+ 30us) server_negotiation.cc:407] Connection header received
0925 02:27:19.020387 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:19.020391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:19.020427 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:19.020576 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:19.021240 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.022467 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.023475 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.024036 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.024130 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:19.026699 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:19.026702 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:19.026702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:19.026713 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:19.028356 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.028709 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.028712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.028712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.028800 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.028956 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.028957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.028957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.029197 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:19.029322 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:19.029705 (+ 383us) server_negotiation.cc:299] Negotiation successful
0925 02:27:19.029816 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
I20260925 02:27:19.032719 15802 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:19.033552 15802 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:19.039103 15802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a51063d188a948d09e9db14fe4238ad8: Generated new TSK 0
I20260925 02:27:19.039264 15802 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:19.064422 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33089
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:19.136327 15816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:19.136543 15816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:19.136580 15816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:19.137619 15816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:19.137676 15816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:19.137718 15816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:19.139063 15816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:19.139289 15816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:19.139488 15816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:19.143827 15824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:19.143872 15822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:19.143955 15816 server_base.cc:1034] running on GCE node
W20260925 02:27:19.143842 15821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:19.144236 15816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:19.144543 15816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:19.145704 15816 hybrid_clock.cc:648] HybridClock initialized: now 1790303239145701 us; error 31 us; skew 500 ppm
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303239, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:19.147852 15816 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:19.148219 15816 webserver.cc:460] Webserver started at http://127.11.220.193:45973/ using document root <none> and password file <none>
I20260925 02:27:19.148435 15816 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:19.148527 15816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:19.148648 15816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:19.149524 15816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "d3c40eabedee4adcb2953f9b38943411"
format_stamp: "Formatted at 2026-09-25 02:27:19 on dist-test-slave-dn4n"
server_key: "cd9502d27dd02407c460ab8df37b4510"
server_key_iv: "3130d553f6e706ea043426fa470ae9d3"
server_key_version: "encryptionkey@0"
I20260925 02:27:19.149852 15816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "d3c40eabedee4adcb2953f9b38943411"
format_stamp: "Formatted at 2026-09-25 02:27:19 on dist-test-slave-dn4n"
server_key: "cd9502d27dd02407c460ab8df37b4510"
server_key_iv: "3130d553f6e706ea043426fa470ae9d3"
server_key_version: "encryptionkey@0"
I20260925 02:27:19.151037 15816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:27:19.151682 15832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:19.151841 15816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:19.151932 15816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "d3c40eabedee4adcb2953f9b38943411"
format_stamp: "Formatted at 2026-09-25 02:27:19 on dist-test-slave-dn4n"
server_key: "cd9502d27dd02407c460ab8df37b4510"
server_key_iv: "3130d553f6e706ea043426fa470ae9d3"
server_key_version: "encryptionkey@0"
I20260925 02:27:19.152009 15816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:19.168282 15816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:19.169836 15816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:19.169981 15816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:19.170186 15816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:19.170456 15816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:19.170506 15816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:19.170542 15816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:19.170567 15816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:19.175679 15816 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:41995
I20260925 02:27:19.175747 15948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:41995 every 8 connection(s)
I20260925 02:27:19.176018 15816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:19.178212 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 15816
I20260925 02:27:19.178308 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:19.178480 12147 external_mini_cluster.cc:1428] Setting key e7bf28f857fa0e2dee4a81a7d9516f3a
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[15707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303239, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:19.185662 15951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:19.176742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33089 (local address 127.11.220.193:47771)
0925 02:27:19.176875 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:19.176906 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:19.177661 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:19.177887 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:19.177889 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:19.178206 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:19.178475 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:19.178481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.179273 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.179274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:19.180488 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:19.180492 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.180993 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.180994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:19.181083 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:19.181580 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:19.181593 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:19.182753 (+ 1160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:19.184499 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:19.184505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:19.184506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:19.184695 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:19.184892 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:19.184893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:19.184893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:19.184977 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:19.185251 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:19.185254 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:19.185361 (+ 107us) client_negotiation.cc:769] Sending connection context
0925 02:27:19.185430 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:27:19.185529 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260925 02:27:19.185678 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:19.176755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:47771 (local address 127.11.220.254:33089)
0925 02:27:19.176799 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:19.176801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:19.176988 (+ 187us) server_negotiation.cc:407] Connection header received
0925 02:27:19.177742 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:19.177743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:19.177776 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:19.177834 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:19.178567 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.179188 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.180567 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.180904 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.180977 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:19.182882 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:19.182884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:19.182885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:19.182893 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:19.184399 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.184765 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.184767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.184767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.184829 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.185034 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.185035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.185035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.185172 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:19.185256 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:19.185481 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 02:27:19.185561 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:19.186142 15949 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33089
I20260925 02:27:19.186226 15949 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:19.186400 15949 heartbeater.cc:511] Master 127.11.220.254:33089 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
I20260925 02:27:19.186885 15753 ts_manager.cc:194] Registered new tserver with Master: d3c40eabedee4adcb2953f9b38943411 (127.11.220.193:41995)
I20260925 02:27:19.187410 15753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:47771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:19.201036 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33089
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:19.275779 15954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:19.275952 15954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:19.275974 15954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:19.277103 15954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:19.277140 15954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:19.277184 15954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:19.278488 15954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:19.278640 15954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:19.278857 15954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:19.283268 15960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:19.283282 15959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:19.283268 15962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:19.283485 15954 server_base.cc:1034] running on GCE node
I20260925 02:27:19.283680 15954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:19.283896 15954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:19.285051 15954 hybrid_clock.cc:648] HybridClock initialized: now 1790303239285032 us; error 30 us; skew 500 ppm
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303239, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:19.287086 15954 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:19.287489 15954 webserver.cc:460] Webserver started at http://127.11.220.194:36213/ using document root <none> and password file <none>
I20260925 02:27:19.287708 15954 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:19.287770 15954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:19.287909 15954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:19.288872 15954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "59011d1a5bcf42b1a3677f39005caddf"
format_stamp: "Formatted at 2026-09-25 02:27:19 on dist-test-slave-dn4n"
server_key: "fc3596bc448c58ea1721f4620ac68f3d"
server_key_iv: "7447e5ff583a4a6621e29f82bc8f974c"
server_key_version: "encryptionkey@0"
I20260925 02:27:19.289191 15954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "59011d1a5bcf42b1a3677f39005caddf"
format_stamp: "Formatted at 2026-09-25 02:27:19 on dist-test-slave-dn4n"
server_key: "fc3596bc448c58ea1721f4620ac68f3d"
server_key_iv: "7447e5ff583a4a6621e29f82bc8f974c"
server_key_version: "encryptionkey@0"
I20260925 02:27:19.290329 15954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:19.290946 15969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:19.291096 15954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:19.291188 15954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "59011d1a5bcf42b1a3677f39005caddf"
format_stamp: "Formatted at 2026-09-25 02:27:19 on dist-test-slave-dn4n"
server_key: "fc3596bc448c58ea1721f4620ac68f3d"
server_key_iv: "7447e5ff583a4a6621e29f82bc8f974c"
server_key_version: "encryptionkey@0"
I20260925 02:27:19.291260 15954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:19.310626 15954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:19.312060 15954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:19.312199 15954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:19.312391 15954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:19.312716 15954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:19.312767 15954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:19.312809 15954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:19.312835 15954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:19.318312 15954 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:35307
I20260925 02:27:19.318347 16083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:35307 every 8 connection(s)
I20260925 02:27:19.318646 15954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[15707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303239, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:19.324854 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 15954
I20260925 02:27:19.324961 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:19.325155 12147 external_mini_cluster.cc:1428] Setting key d61fbc966ea672c03d0bde4820eca517
I20260925 02:27:19.327196 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:19.319284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:36241 (local address 127.11.220.254:33089)
0925 02:27:19.319347 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:19.319349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:19.319513 (+ 164us) server_negotiation.cc:407] Connection header received
0925 02:27:19.320304 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:19.320305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:19.320314 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:19.320364 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:19.320943 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.321440 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.322371 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.322535 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.322580 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:19.324338 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:19.324339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:19.324340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:19.324345 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:19.325984 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.326342 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.326343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.326343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.326398 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.326638 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.326639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.326639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.326752 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:19.326817 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:19.327042 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 02:27:19.327120 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:27:19.327339 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:19.319254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33089 (local address 127.11.220.194:36241)
0925 02:27:19.319402 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:19.319438 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:19.320225 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:19.320409 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:19.320411 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:19.320636 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:19.320793 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:19.320796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.321524 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.321525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:19.322288 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:19.322290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.322601 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.322602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:19.322683 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:19.323061 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:19.323068 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:19.324261 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:19.326057 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:19.326063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:19.326065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:19.326259 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:19.326457 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:19.326458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:19.326458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:19.326559 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:19.326810 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:19.326817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:19.326932 (+ 115us) client_negotiation.cc:769] Sending connection context
0925 02:27:19.327024 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 02:27:19.327149 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260925 02:27:19.327715 16084 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33089
I20260925 02:27:19.327790 16084 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:19.327945 16084 heartbeater.cc:511] Master 127.11.220.254:33089 requested a full tablet report, sending...
I20260925 02:27:19.328303 15753 ts_manager.cc:194] Registered new tserver with Master: 59011d1a5bcf42b1a3677f39005caddf (127.11.220.194:35307)
I20260925 02:27:19.328692 15753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:36241
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:19.346885 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33089
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:19.419147 16089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:19.419322 16089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:19.419342 16089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:19.420466 16089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:19.420506 16089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:19.420531 16089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:19.422102 16089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:19.422353 16089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:19.422576 16089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:19.427059 16094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:19.427081 16095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:19.427100 16097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:19.427251 16089 server_base.cc:1034] running on GCE node
I20260925 02:27:19.427507 16089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:19.427723 16089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:19.428879 16089 hybrid_clock.cc:648] HybridClock initialized: now 1790303239428870 us; error 37 us; skew 500 ppm
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303239, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:19.431073 16089 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:19.431445 16089 webserver.cc:460] Webserver started at http://127.11.220.195:45305/ using document root <none> and password file <none>
I20260925 02:27:19.431654 16089 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:19.431715 16089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:19.431859 16089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:19.432825 16089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "004f0804546e4d5d90a6cc3e5497cd86"
format_stamp: "Formatted at 2026-09-25 02:27:19 on dist-test-slave-dn4n"
server_key: "875ccec54d2b23d306b021b74380e8cc"
server_key_iv: "dab40435e72e839ab0925c09e89b0933"
server_key_version: "encryptionkey@0"
I20260925 02:27:19.433158 16089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "004f0804546e4d5d90a6cc3e5497cd86"
format_stamp: "Formatted at 2026-09-25 02:27:19 on dist-test-slave-dn4n"
server_key: "875ccec54d2b23d306b021b74380e8cc"
server_key_iv: "dab40435e72e839ab0925c09e89b0933"
server_key_version: "encryptionkey@0"
I20260925 02:27:19.434304 16089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:19.434942 16104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:19.435087 16089 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:19.435194 16089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "004f0804546e4d5d90a6cc3e5497cd86"
format_stamp: "Formatted at 2026-09-25 02:27:19 on dist-test-slave-dn4n"
server_key: "875ccec54d2b23d306b021b74380e8cc"
server_key_iv: "dab40435e72e839ab0925c09e89b0933"
server_key_version: "encryptionkey@0"
I20260925 02:27:19.435268 16089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:19.446682 16089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:19.448778 16089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:19.448948 16089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:19.449141 16089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:19.449465 16089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:19.449524 16089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:19.449556 16089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:19.449581 16089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:19.454844 16089 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:41477
I20260925 02:27:19.454874 16218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:41477 every 8 connection(s)
I20260925 02:27:19.455123 16089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[15707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303239, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:19.460601 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 16089
I20260925 02:27:19.460681 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:19.460891 12147 external_mini_cluster.cc:1428] Setting key ad76e4ef670109f92c9a0b9d69aac2e6
I20260925 02:27:19.463021 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:19.455800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:42767 (local address 127.11.220.254:33089)
0925 02:27:19.455859 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:19.455861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:19.456011 (+ 150us) server_negotiation.cc:407] Connection header received
0925 02:27:19.456821 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:19.456823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:19.456835 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:19.456885 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:19.457402 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.457845 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.458701 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.458882 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.458945 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:19.460805 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:19.460807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:19.460807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:19.460815 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:19.462052 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.462366 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.462366 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.462367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.462410 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.462585 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.462586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.462586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.462667 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:19.462707 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:19.462909 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 02:27:19.462962 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:27:19.463171 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:19.455718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33089 (local address 127.11.220.195:42767)
0925 02:27:19.455893 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:19.455929 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:19.456720 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:19.456941 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:19.456944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:19.457172 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:19.457315 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:19.457319 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.457929 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.457929 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:19.458601 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:19.458604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.458953 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.458954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:19.459062 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:19.459424 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:19.459432 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:19.460698 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:19.462111 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:19.462116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:19.462118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:19.462307 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:19.462459 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:19.462459 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:19.462459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:19.462540 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:19.462713 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:19.462716 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:19.462809 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 02:27:19.462896 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 02:27:19.462999 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
I20260925 02:27:19.463564 16219 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33089
I20260925 02:27:19.463665 16219 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:19.463819 16219 heartbeater.cc:511] Master 127.11.220.254:33089 requested a full tablet report, sending...
I20260925 02:27:19.464159 15753 ts_manager.cc:194] Registered new tserver with Master: 004f0804546e4d5d90a6cc3e5497cd86 (127.11.220.195:41477)
I20260925 02:27:19.464510 15753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:42767
I20260925 02:27:19.472797 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[15707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[15707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:27:19.492020 16226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:19.485716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48164 (local address 127.11.220.193:41995)
0925 02:27:19.485897 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:19.485901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:19.485909 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:19.485929 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:19.485931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:19.485942 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:19.485998 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:19.486476 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.487085 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.487694 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.487972 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.488019 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:19.489907 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:19.489914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:19.489914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:19.489920 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:19.491167 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.491425 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.491426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.491427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.491473 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.491597 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.491597 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.491597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.491693 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:19.491798 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:19.491859 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 02:27:19.491911 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[15707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:19.498880 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:19.493155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39528 (local address 127.11.220.254:33089)
0925 02:27:19.493190 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:19.493192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:19.493211 (+ 19us) server_negotiation.cc:407] Connection header received
0925 02:27:19.493234 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:19.493235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:19.493242 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:19.493293 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:19.493690 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.494117 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.494687 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:19.494915 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:19.494983 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:19.496770 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:19.496771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:19.496772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:19.496777 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:19.497836 (+ 1059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.498142 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.498143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.498143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.498192 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:19.498395 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:19.498396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:19.498396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:19.498526 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:19.498614 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:19.498715 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 02:27:19.498789 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 02:27:19.499049 15753 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39528
I20260925 02:27:19.499321 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 15816
I20260925 02:27:19.504673 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 15954
I20260925 02:27:19.508725 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 16089
I20260925 02:27:19.512583 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 15720
2026-09-25T02:27:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1639 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[16231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[16231](info): set up 2 sockets
Sep 25 02:27:19 dist-test-slave-dn4n krb5kdc[16231](info): commencing operation
krb5kdc: starting...
W20260925 02:27:20.448612 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:20 dist-test-slave-dn4n krb5kdc[16231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:20.548414 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:34901
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:34901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:20.621042 16244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:20.621240 16244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:20.621280 16244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:20.622366 16244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:20.622423 16244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:20.622447 16244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:20.622478 16244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:20.622502 16244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:20.623885 16244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:34901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:34901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:20.624147 16244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:20.624346 16244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:20.628587 16250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:20.628597 16252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:20.628598 16249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:20.628731 16244 server_base.cc:1034] running on GCE node
I20260925 02:27:20.629027 16244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:20.629294 16244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:20.630447 16244 hybrid_clock.cc:648] HybridClock initialized: now 1790303240630431 us; error 27 us; skew 500 ppm
Sep 25 02:27:20 dist-test-slave-dn4n krb5kdc[16231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303240, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:20.632591 16244 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:20.632942 16244 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:20.633028 16244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:20.633155 16244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:20.634079 16244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "375f7662a1a847beb2a4bea73b84a45a"
format_stamp: "Formatted at 2026-09-25 02:27:20 on dist-test-slave-dn4n"
server_key: "4b514171d9df01e1c7cb3470aaf05ceb"
server_key_iv: "e34dfa5f94af4c021efe7c088c4d7bb1"
server_key_version: "encryptionkey@0"
I20260925 02:27:20.634455 16244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "375f7662a1a847beb2a4bea73b84a45a"
format_stamp: "Formatted at 2026-09-25 02:27:20 on dist-test-slave-dn4n"
server_key: "4b514171d9df01e1c7cb3470aaf05ceb"
server_key_iv: "e34dfa5f94af4c021efe7c088c4d7bb1"
server_key_version: "encryptionkey@0"
I20260925 02:27:20.635571 16244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:20.636183 16258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:20.636340 16244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:20.636427 16244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "375f7662a1a847beb2a4bea73b84a45a"
format_stamp: "Formatted at 2026-09-25 02:27:20 on dist-test-slave-dn4n"
server_key: "4b514171d9df01e1c7cb3470aaf05ceb"
server_key_iv: "e34dfa5f94af4c021efe7c088c4d7bb1"
server_key_version: "encryptionkey@0"
I20260925 02:27:20.636511 16244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:20.650341 16244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:20.651814 16244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:20.651966 16244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:20.655933 16244 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:34901
I20260925 02:27:20.655953 16311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:34901 every 8 connection(s)
I20260925 02:27:20.656206 16244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:20.657043 16312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:20.658900 16312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a: Bootstrap starting.
I20260925 02:27:20.659336 16312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:20.659544 16312 log.cc:824] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:20.660061 16312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a: No bootstrap required, opened a new log
I20260925 02:27:20.661564 16312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375f7662a1a847beb2a4bea73b84a45a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34901 } }
I20260925 02:27:20.661686 16312 raft_consensus.cc:374] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:20.661707 16312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375f7662a1a847beb2a4bea73b84a45a, State: Initialized, Role: FOLLOWER
I20260925 02:27:20.661791 16312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375f7662a1a847beb2a4bea73b84a45a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34901 } }
I20260925 02:27:20.661859 16312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:20.661892 16312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:20.661940 16312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:20.662168 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 16244
I20260925 02:27:20.662262 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:20.662427 12147 external_mini_cluster.cc:1428] Setting key 617b6b5bf3f52bcbede11e5a80da76c1
I20260925 02:27:20.662873 16312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375f7662a1a847beb2a4bea73b84a45a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34901 } }
I20260925 02:27:20.662995 16312 leader_election.cc:302] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 375f7662a1a847beb2a4bea73b84a45a; no voters:
I20260925 02:27:20.663151 16312 leader_election.cc:288] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:20.663209 16317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:20.663311 16317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [term 1 LEADER]: Becoming Leader. State: Replica: 375f7662a1a847beb2a4bea73b84a45a, State: Running, Role: LEADER
I20260925 02:27:20.663410 16312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:20.663398 16317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375f7662a1a847beb2a4bea73b84a45a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34901 } }
I20260925 02:27:20.664772 16318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375f7662a1a847beb2a4bea73b84a45a. Latest consensus state: current_term: 1 leader_uuid: "375f7662a1a847beb2a4bea73b84a45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375f7662a1a847beb2a4bea73b84a45a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34901 } } }
I20260925 02:27:20.664857 16318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:20.665194 16319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "375f7662a1a847beb2a4bea73b84a45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375f7662a1a847beb2a4bea73b84a45a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34901 } } }
I20260925 02:27:20.665266 16319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a [sys.catalog]: This master's current role is: LEADER
W20260925 02:27:20.665347 16331 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:27:20.665398 16331 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:27:20.665463 16332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:20.665622 16332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:20.666918 16332 catalog_manager.cc:1347] Generated new cluster ID: 753160994515411f82e2ca55cd4adb76
I20260925 02:27:20.666965 16332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:27:20 dist-test-slave-dn4n krb5kdc[16231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:20.673648 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:20.662842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60190 (local address 127.11.220.254:34901)
0925 02:27:20.663089 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:20.663097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:20.663125 (+ 28us) server_negotiation.cc:407] Connection header received
0925 02:27:20.663257 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:20.663263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:20.663315 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:20.663497 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:20.664325 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:20.665762 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:20.666777 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:20.667324 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:20.667426 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:20.669906 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:20.669910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:20.669910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:20.669921 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:20.672144 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:20.672510 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:20.672513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:20.672514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:20.672619 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:20.672801 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:20.672802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:20.672802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:20.673093 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:20.673259 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:20.673339 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 02:27:20.673466 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":124,"threads_started":1}
I20260925 02:27:20.679242 16332 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:20.679896 16332 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:20.686092 16332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a: Generated new TSK 0
I20260925 02:27:20.686240 16332 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:20.710798 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34901
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:20.784109 16338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:20.784299 16338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:20.784350 16338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:20.785413 16338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:20.785447 16338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:20.785470 16338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:20.786777 16338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:20.786932 16338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:20.787072 16338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:20.791383 16343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:20.791350 16346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:20.791361 16344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:20.791555 16338 server_base.cc:1034] running on GCE node
I20260925 02:27:20.791738 16338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:20.791908 16338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:20.793035 16338 hybrid_clock.cc:648] HybridClock initialized: now 1790303240793024 us; error 26 us; skew 500 ppm
Sep 25 02:27:20 dist-test-slave-dn4n krb5kdc[16231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303240, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:20.795176 16338 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:20.795490 16338 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:20.795553 16338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:20.795686 16338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:20.796626 16338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "c4a543b90f364d9984a0f3a24c07482c"
format_stamp: "Formatted at 2026-09-25 02:27:20 on dist-test-slave-dn4n"
server_key: "3f33de9221fcb72bbce2d0c8c90987ea"
server_key_iv: "fd1f075ac52734460cc45ab3eab1f27d"
server_key_version: "encryptionkey@0"
I20260925 02:27:20.796963 16338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "c4a543b90f364d9984a0f3a24c07482c"
format_stamp: "Formatted at 2026-09-25 02:27:20 on dist-test-slave-dn4n"
server_key: "3f33de9221fcb72bbce2d0c8c90987ea"
server_key_iv: "fd1f075ac52734460cc45ab3eab1f27d"
server_key_version: "encryptionkey@0"
I20260925 02:27:20.798076 16338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:20.798699 16352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:20.798852 16338 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:20.798939 16338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "c4a543b90f364d9984a0f3a24c07482c"
format_stamp: "Formatted at 2026-09-25 02:27:20 on dist-test-slave-dn4n"
server_key: "3f33de9221fcb72bbce2d0c8c90987ea"
server_key_iv: "fd1f075ac52734460cc45ab3eab1f27d"
server_key_version: "encryptionkey@0"
I20260925 02:27:20.799010 16338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:20.810595 16338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:20.811966 16338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:20.812101 16338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:20.812291 16338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:20.812593 16338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:20.812640 16338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:20.812668 16338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:20.812695 16338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:20.818226 16338 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:39321
I20260925 02:27:20.818254 16465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:39321 every 8 connection(s)
I20260925 02:27:20.818472 16338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
Sep 25 02:27:20 dist-test-slave-dn4n krb5kdc[16231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303240, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:20.824872 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 16338
I20260925 02:27:20.824990 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:20.825155 12147 external_mini_cluster.cc:1428] Setting key 1519f4b80bd69d0196c8fae2e323adc0
I20260925 02:27:20.826344 16468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:20.819073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:34901 (local address 127.11.220.193:49755)
0925 02:27:20.819206 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:20.819233 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:20.819555 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:20.819740 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:20.819742 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:20.819946 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:20.820366 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:20.820370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:20.820995 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:20.820996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:20.821689 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:20.821691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:20.821988 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:20.821989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:20.822072 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:20.822442 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:20.822449 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:20.823605 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:20.824917 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:20.824924 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:20.824926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:20.825187 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:20.825488 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:20.825489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:20.825489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:20.825600 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:20.825877 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:20.825882 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:20.825976 (+ 94us) client_negotiation.cc:769] Sending connection context
0925 02:27:20.826051 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:27:20.826201 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260925 02:27:20.826360 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:20.819085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:49755 (local address 127.11.220.254:34901)
0925 02:27:20.819125 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:20.819127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:20.819308 (+ 181us) server_negotiation.cc:407] Connection header received
0925 02:27:20.819621 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:20.819622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:20.819632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:20.819681 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:20.820432 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:20.820918 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:20.821755 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:20.821924 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:20.821970 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:20.823685 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:20.823687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:20.823687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:20.823693 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:20.824839 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:20.825313 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:20.825314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:20.825315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:20.825367 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:20.825681 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:20.825682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:20.825683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:20.825812 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:20.825877 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:20.826129 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 02:27:20.826218 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:27:20.826843 16466 heartbeater.cc:347] Connected to a master server at 127.11.220.254:34901
I20260925 02:27:20.826937 16466 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:20.827083 16466 heartbeater.cc:511] Master 127.11.220.254:34901 requested a full tablet report, sending...
I20260925 02:27:20.827646 16275 ts_manager.cc:194] Registered new tserver with Master: c4a543b90f364d9984a0f3a24c07482c (127.11.220.193:39321)
I20260925 02:27:20.828526 16275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:49755
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:20.847198 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34901
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:20.917811 16471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:20.918009 16471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:20.918047 16471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:20.919564 16471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:20.919622 16471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:20.919673 16471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:20.921083 16471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:20.921340 16471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:20.921550 16471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:20.925786 16476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:20.925789 16477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:20.925900 16471 server_base.cc:1034] running on GCE node
W20260925 02:27:20.925782 16479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:20.926147 16471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:20.926370 16471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:20.927531 16471 hybrid_clock.cc:648] HybridClock initialized: now 1790303240927521 us; error 22 us; skew 500 ppm
Sep 25 02:27:20 dist-test-slave-dn4n krb5kdc[16231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303240, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:20.929670 16471 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:20.929970 16471 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:20.930033 16471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:20.930164 16471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:20.931032 16471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "d3d12b9401b6428abbc1b51e625e337a"
format_stamp: "Formatted at 2026-09-25 02:27:20 on dist-test-slave-dn4n"
server_key: "1eb7385cf554236a05c6ea1982d6bf39"
server_key_iv: "5e43687dded85f42da9ed8aef855524f"
server_key_version: "encryptionkey@0"
I20260925 02:27:20.931354 16471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "d3d12b9401b6428abbc1b51e625e337a"
format_stamp: "Formatted at 2026-09-25 02:27:20 on dist-test-slave-dn4n"
server_key: "1eb7385cf554236a05c6ea1982d6bf39"
server_key_iv: "5e43687dded85f42da9ed8aef855524f"
server_key_version: "encryptionkey@0"
I20260925 02:27:20.932659 16471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:20.933295 16486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:20.933450 16471 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:20.933534 16471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "d3d12b9401b6428abbc1b51e625e337a"
format_stamp: "Formatted at 2026-09-25 02:27:20 on dist-test-slave-dn4n"
server_key: "1eb7385cf554236a05c6ea1982d6bf39"
server_key_iv: "5e43687dded85f42da9ed8aef855524f"
server_key_version: "encryptionkey@0"
I20260925 02:27:20.933606 16471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:20.945681 16471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:20.947201 16471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:20.947338 16471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:20.947521 16471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:20.947791 16471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:20.947839 16471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:20.947873 16471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:20.947898 16471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:20.953436 16471 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:35995
I20260925 02:27:20.953526 16603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:35995 every 8 connection(s)
I20260925 02:27:20.953734 16471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:27:20 dist-test-slave-dn4n krb5kdc[16231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303240, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:20.960896 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 16471
I20260925 02:27:20.960994 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:20.961172 12147 external_mini_cluster.cc:1428] Setting key 349d1276df7e09402fecc033a8fc9513
I20260925 02:27:20.963397 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:20.954387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:54371 (local address 127.11.220.254:34901)
0925 02:27:20.954447 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:20.954449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:20.954621 (+ 172us) server_negotiation.cc:407] Connection header received
0925 02:27:20.955044 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:20.955045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:20.955054 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:20.955110 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:20.955972 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:20.956736 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:20.957964 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:20.958327 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:20.958394 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:20.960369 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:20.960370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:20.960371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:20.960377 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:20.962024 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:20.962419 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:20.962432 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:20.962432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:20.962507 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:20.962758 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:20.962759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:20.962759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:20.962869 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:20.962945 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:20.963233 (+ 288us) server_negotiation.cc:299] Negotiation successful
0925 02:27:20.963312 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:27:20.963593 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:20.954346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:34901 (local address 127.11.220.194:54371)
0925 02:27:20.954526 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:20.954560 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:20.954962 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:20.955147 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:20.955150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:20.955370 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:20.955891 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:20.955897 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:20.956854 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:20.956855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:20.957889 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:20.957892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:20.958415 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:20.958416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:20.958542 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:20.959080 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:20.959090 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:20.960290 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:20.962111 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:20.962118 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:20.962120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:20.962333 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:20.962572 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:20.962573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:20.962573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:20.962682 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:20.962953 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:20.962960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:20.963096 (+ 136us) client_negotiation.cc:769] Sending connection context
0925 02:27:20.963208 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 02:27:20.963347 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260925 02:27:20.964026 16604 heartbeater.cc:347] Connected to a master server at 127.11.220.254:34901
I20260925 02:27:20.964179 16604 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:20.964308 16604 heartbeater.cc:511] Master 127.11.220.254:34901 requested a full tablet report, sending...
I20260925 02:27:20.964669 16275 ts_manager.cc:194] Registered new tserver with Master: d3d12b9401b6428abbc1b51e625e337a (127.11.220.194:35995)
I20260925 02:27:20.965013 16275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:54371
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:20.982909 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34901
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:21.058797 16609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:21.059002 16609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:21.059033 16609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:21.060171 16609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:21.060230 16609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:21.060283 16609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:21.061775 16609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:21.062011 16609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:21.062204 16609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:21.066550 16617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.066556 16615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.066556 16614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:21.066772 16609 server_base.cc:1034] running on GCE node
I20260925 02:27:21.066994 16609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:21.067206 16609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:21.068367 16609 hybrid_clock.cc:648] HybridClock initialized: now 1790303241068358 us; error 32 us; skew 500 ppm
Sep 25 02:27:21 dist-test-slave-dn4n krb5kdc[16231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303241, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:21.070731 16609 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:21.071062 16609 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:21.071110 16609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:21.071192 16609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:21.072039 16609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "b8e8017a7f634a44ac85e4ddc308b261"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "5ab548a9d58cd5ac74f45c3d10c3bb39"
server_key_iv: "9903014c434f974dac2352606efa48eb"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.072293 16609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "b8e8017a7f634a44ac85e4ddc308b261"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "5ab548a9d58cd5ac74f45c3d10c3bb39"
server_key_iv: "9903014c434f974dac2352606efa48eb"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.073485 16609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:21.074084 16623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.074275 16609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:21.074334 16609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "b8e8017a7f634a44ac85e4ddc308b261"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "5ab548a9d58cd5ac74f45c3d10c3bb39"
server_key_iv: "9903014c434f974dac2352606efa48eb"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.074416 16609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:21.091267 16609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:21.093174 16609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:21.093264 16609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:21.093400 16609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:21.093660 16609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:21.093698 16609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.093736 16609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:21.093760 16609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.098927 16609 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:44371
I20260925 02:27:21.098966 16738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:44371 every 8 connection(s)
I20260925 02:27:21.099172 16609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:27:21 dist-test-slave-dn4n krb5kdc[16231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303241, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:21.106858 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 16609
I20260925 02:27:21.106958 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:21.107079 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.099757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:41585 (local address 127.11.220.254:34901)
0925 02:27:21.099826 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.099829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.099952 (+ 123us) server_negotiation.cc:407] Connection header received
0925 02:27:21.100356 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.100357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.100367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.100411 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.101252 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.101712 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.102702 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.102878 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.102928 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.104684 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.104686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.104686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:21.104691 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.105974 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:21.106282 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:21.106282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:21.106283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:21.106327 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:21.106504 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:21.106504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:21.106505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:21.106587 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.106631 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.106883 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.106977 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:27:21.107134 12147 external_mini_cluster.cc:1428] Setting key 709f6283ffa6ff865ede76173ae99113
I20260925 02:27:21.107226 16741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.099711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:34901 (local address 127.11.220.195:41585)
0925 02:27:21.099856 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.099884 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.100281 (+ 397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.100510 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.100514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.100752 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.101182 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.101187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.101786 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.101786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.102636 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.102639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.102945 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.102946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.103037 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.103399 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:21.103405 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.104599 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.106033 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:21.106037 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:21.106039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:21.106219 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:21.106378 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:21.106379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:21.106379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:21.106457 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:21.106634 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.106637 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.106759 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.106864 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.107012 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260925 02:27:21.107695 16739 heartbeater.cc:347] Connected to a master server at 127.11.220.254:34901
I20260925 02:27:21.107796 16739 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:21.107920 16739 heartbeater.cc:511] Master 127.11.220.254:34901 requested a full tablet report, sending...
I20260925 02:27:21.108275 16275 ts_manager.cc:194] Registered new tserver with Master: b8e8017a7f634a44ac85e4ddc308b261 (127.11.220.195:44371)
I20260925 02:27:21.108636 16275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:41585
I20260925 02:27:21.118731 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:27:21.124894 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.119891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.11.220.254:34901)
0925 02:27:21.119940 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.119941 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.120012 (+ 71us) server_negotiation.cc:407] Connection header received
0925 02:27:21.120056 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.120057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.120064 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.120115 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.120638 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.121022 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.121634 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.121783 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.121825 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.122732 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.122734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.122734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:21.122740 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.123791 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:21.124111 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:21.124112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:21.124113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:21.124161 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:21.124357 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:21.124358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:21.124358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:21.124552 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.124646 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.124744 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.124810 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:27:21.125996 16275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:21.127404 16275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:21.133080 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.129375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:39321 (local address 127.0.0.1:32870)
0925 02:27:21.129686 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.129693 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.129731 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.130013 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.130014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.130016 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:21.130055 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.130057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.131251 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.131252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.132035 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.132037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.132818 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.132819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.132877 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.132881 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.132906 (+ 25us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.132956 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":47,"thread_start_us":79,"threads_started":1}
I20260925 02:27:21.133085 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.129424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.11.220.195:44371)
0925 02:27:21.129546 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.129550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.129573 (+ 23us) server_negotiation.cc:407] Connection header received
0925 02:27:21.129687 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.129690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.129707 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.129767 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:21.129975 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.130808 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.131790 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.132699 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.132776 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.132823 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.132970 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.133019 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":74,"threads_started":1}
I20260925 02:27:21.133288 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.129690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.11.220.193:39321)
0925 02:27:21.129844 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.129849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.129862 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:27:21.129890 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.129892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.129902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.129987 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:21.130129 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.131139 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.132117 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.132731 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.133058 (+ 327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.133116 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.133141 (+ 25us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.133194 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260925 02:27:21.133576 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.129352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:44371 (local address 127.0.0.1:33054)
0925 02:27:21.129492 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.129523 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.129586 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.129844 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.129846 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.129848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:21.129896 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.129898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.130877 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.130878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.131703 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.131705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.132776 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.132777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.132850 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.132862 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.132951 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.132990 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":49,"threads_started":1}
I20260925 02:27:21.133749 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.129544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:35995 (local address 127.0.0.1:50190)
0925 02:27:21.130095 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.130104 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.130145 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.130352 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.130353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.130355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:21.130390 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.130391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.131451 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.131452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.132432 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.132434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.133511 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.133513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.133556 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.133559 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.133643 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.133687 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":90,"threads_started":1}
I20260925 02:27:21.133774 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.129658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50190 (local address 127.11.220.194:35995)
0925 02:27:21.129851 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.129855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.130198 (+ 343us) server_negotiation.cc:407] Connection header received
0925 02:27:21.130238 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.130239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.130252 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.130335 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:21.130449 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.131361 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.132541 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.133206 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.133270 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.133314 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.133664 (+ 350us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.133699 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":77,"threads_started":1}
I20260925 02:27:21.133966 16400 tablet_service.cc:1467] Processing CreateTablet for tablet 6739072b86cf40b9b4e4f3da9da91643 (DEFAULT_TABLE table=test-table [id=fc75aadfd6cb451f9086fd21d2e404d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:21.134310 16400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6739072b86cf40b9b4e4f3da9da91643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:21.134377 16534 tablet_service.cc:1467] Processing CreateTablet for tablet 6739072b86cf40b9b4e4f3da9da91643 (DEFAULT_TABLE table=test-table [id=fc75aadfd6cb451f9086fd21d2e404d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:21.134646 16534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6739072b86cf40b9b4e4f3da9da91643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:21.137010 16672 tablet_service.cc:1467] Processing CreateTablet for tablet 6739072b86cf40b9b4e4f3da9da91643 (DEFAULT_TABLE table=test-table [id=fc75aadfd6cb451f9086fd21d2e404d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:21.137281 16672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6739072b86cf40b9b4e4f3da9da91643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:21.137420 16753 tablet_bootstrap.cc:492] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c: Bootstrap starting.
I20260925 02:27:21.137550 16754 tablet_bootstrap.cc:492] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: Bootstrap starting.
I20260925 02:27:21.138036 16754 tablet_bootstrap.cc:654] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:21.138110 16753 tablet_bootstrap.cc:654] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:21.138558 16754 log.cc:824] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:21.138747 16753 log.cc:824] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:21.139289 16754 tablet_bootstrap.cc:492] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: No bootstrap required, opened a new log
I20260925 02:27:21.139356 16754 ts_tablet_manager.cc:1397] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:21.139482 16753 tablet_bootstrap.cc:492] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c: No bootstrap required, opened a new log
I20260925 02:27:21.139550 16753 ts_tablet_manager.cc:1397] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:21.140331 16757 tablet_bootstrap.cc:492] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: Bootstrap starting.
I20260925 02:27:21.140938 16757 tablet_bootstrap.cc:654] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:21.141242 16754 raft_consensus.cc:348] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 } } peers { permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 } } peers { permanent_uuid: "d3d12b9401b6428abbc1b51e625e337a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35995 } }
I20260925 02:27:21.141377 16754 raft_consensus.cc:374] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:21.141417 16754 raft_consensus.cc:729] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3d12b9401b6428abbc1b51e625e337a, State: Initialized, Role: FOLLOWER
I20260925 02:27:21.141343 16753 raft_consensus.cc:348] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 } } peers { permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 } } peers { permanent_uuid: "d3d12b9401b6428abbc1b51e625e337a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35995 } }
I20260925 02:27:21.141490 16757 log.cc:824] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:21.141511 16753 raft_consensus.cc:374] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:21.141551 16753 raft_consensus.cc:729] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4a543b90f364d9984a0f3a24c07482c, State: Initialized, Role: FOLLOWER
I20260925 02:27:21.141511 16754 consensus_queue.cc:260] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 } } peers { permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 } } peers { permanent_uuid: "d3d12b9401b6428abbc1b51e625e337a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35995 } }
I20260925 02:27:21.141670 16753 consensus_queue.cc:260] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 } } peers { permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 } } peers { permanent_uuid: "d3d12b9401b6428abbc1b51e625e337a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35995 } }
I20260925 02:27:21.141781 16754 ts_tablet_manager.cc:1428] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:27:21.141793 16604 heartbeater.cc:503] Master 127.11.220.254:34901 was elected leader, sending a full tablet report...
I20260925 02:27:21.141920 16753 ts_tablet_manager.cc:1428] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:27:21.142017 16466 heartbeater.cc:503] Master 127.11.220.254:34901 was elected leader, sending a full tablet report...
I20260925 02:27:21.142530 16757 tablet_bootstrap.cc:492] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: No bootstrap required, opened a new log
I20260925 02:27:21.142590 16757 ts_tablet_manager.cc:1397] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:21.143769 16757 raft_consensus.cc:348] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 } } peers { permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 } } peers { permanent_uuid: "d3d12b9401b6428abbc1b51e625e337a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35995 } }
I20260925 02:27:21.143898 16757 raft_consensus.cc:374] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:21.143940 16757 raft_consensus.cc:729] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e8017a7f634a44ac85e4ddc308b261, State: Initialized, Role: FOLLOWER
I20260925 02:27:21.144044 16757 consensus_queue.cc:260] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 } } peers { permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 } } peers { permanent_uuid: "d3d12b9401b6428abbc1b51e625e337a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35995 } }
I20260925 02:27:21.144307 16757 ts_tablet_manager.cc:1428] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:27:21.144323 16739 heartbeater.cc:503] Master 127.11.220.254:34901 was elected leader, sending a full tablet report...
I20260925 02:27:21.162230 16759 raft_consensus.cc:482] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:21.162314 16759 raft_consensus.cc:504] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 } } peers { permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 } } peers { permanent_uuid: "d3d12b9401b6428abbc1b51e625e337a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35995 } }
I20260925 02:27:21.162563 16759 leader_election.cc:288] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e8017a7f634a44ac85e4ddc308b261 (127.11.220.195:44371), c4a543b90f364d9984a0f3a24c07482c (127.11.220.193:39321)
I20260925 02:27:21.165308 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.162683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:44371 (local address 127.11.220.194:36913)
0925 02:27:21.162810 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.162822 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.162880 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.163019 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.163020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.163021 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:21.163096 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.163097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.163657 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.163659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.164439 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.164441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.165061 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.165062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.165126 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.165130 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.165190 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.165227 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":10,"thread_start_us":73,"threads_started":1}
I20260925 02:27:21.165313 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.162739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:36913 (local address 127.11.220.195:44371)
0925 02:27:21.162778 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.162779 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.162875 (+ 96us) server_negotiation.cc:407] Connection header received
0925 02:27:21.162947 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.162947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.162955 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.163017 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:21.163152 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.163542 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.164566 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.165002 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.165058 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.165063 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.165223 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.165258 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 02:27:21.165545 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.162680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:39321 (local address 127.11.220.194:45011)
0925 02:27:21.162720 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.162729 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.162760 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.162890 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.162891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.162893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:21.162974 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.162975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.163625 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.163626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.164586 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.164588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.165301 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.165302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.165381 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.165384 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.165449 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.165487 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":18}
I20260925 02:27:21.165594 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.162753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:45011 (local address 127.11.220.193:39321)
0925 02:27:21.162783 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.162784 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.162792 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:21.162814 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.162814 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.162824 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.162887 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:21.163071 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.163538 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.164661 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.165206 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.165256 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.165263 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.165489 (+ 226us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.165534 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:21.165550 16692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6739072b86cf40b9b4e4f3da9da91643" candidate_uuid: "d3d12b9401b6428abbc1b51e625e337a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e8017a7f634a44ac85e4ddc308b261" is_pre_election: true
I20260925 02:27:21.165685 16692 raft_consensus.cc:2454] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3d12b9401b6428abbc1b51e625e337a in term 0.
I20260925 02:27:21.165860 16487 leader_election.cc:302] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e8017a7f634a44ac85e4ddc308b261, d3d12b9401b6428abbc1b51e625e337a; no voters:
I20260925 02:27:21.165849 16420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6739072b86cf40b9b4e4f3da9da91643" candidate_uuid: "d3d12b9401b6428abbc1b51e625e337a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a543b90f364d9984a0f3a24c07482c" is_pre_election: true
I20260925 02:27:21.165954 16420 raft_consensus.cc:2454] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3d12b9401b6428abbc1b51e625e337a in term 0.
I20260925 02:27:21.165968 16759 raft_consensus.cc:2781] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:21.166007 16759 raft_consensus.cc:482] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:21.166034 16759 raft_consensus.cc:3037] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:21.166559 16759 raft_consensus.cc:504] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 } } peers { permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 } } peers { permanent_uuid: "d3d12b9401b6428abbc1b51e625e337a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35995 } }
I20260925 02:27:21.166694 16759 leader_election.cc:288] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [CANDIDATE]: Term 1 election: Requested vote from peers b8e8017a7f634a44ac85e4ddc308b261 (127.11.220.195:44371), c4a543b90f364d9984a0f3a24c07482c (127.11.220.193:39321)
I20260925 02:27:21.166841 16420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6739072b86cf40b9b4e4f3da9da91643" candidate_uuid: "d3d12b9401b6428abbc1b51e625e337a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4a543b90f364d9984a0f3a24c07482c"
I20260925 02:27:21.166891 16420 raft_consensus.cc:3037] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:21.167459 16420 raft_consensus.cc:2454] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3d12b9401b6428abbc1b51e625e337a in term 1.
I20260925 02:27:21.167570 16692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6739072b86cf40b9b4e4f3da9da91643" candidate_uuid: "d3d12b9401b6428abbc1b51e625e337a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e8017a7f634a44ac85e4ddc308b261"
I20260925 02:27:21.167640 16692 raft_consensus.cc:3037] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:21.167629 16489 leader_election.cc:302] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4a543b90f364d9984a0f3a24c07482c, d3d12b9401b6428abbc1b51e625e337a; no voters:
I20260925 02:27:21.167721 16759 raft_consensus.cc:2781] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:21.167871 16759 raft_consensus.cc:686] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 1 LEADER]: Becoming Leader. State: Replica: d3d12b9401b6428abbc1b51e625e337a, State: Running, Role: LEADER
I20260925 02:27:21.167973 16759 consensus_queue.cc:237] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 } } peers { permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 } } peers { permanent_uuid: "d3d12b9401b6428abbc1b51e625e337a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35995 } }
I20260925 02:27:21.168475 16692 raft_consensus.cc:2454] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3d12b9401b6428abbc1b51e625e337a in term 1.
I20260925 02:27:21.168573 16275 catalog_manager.cc:5495] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a reported cstate change: term changed from 0 to 1, leader changed from <none> to d3d12b9401b6428abbc1b51e625e337a (127.11.220.194). New cstate: current_term: 1 leader_uuid: "d3d12b9401b6428abbc1b51e625e337a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3d12b9401b6428abbc1b51e625e337a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35995 } health_report { overall_health: HEALTHY } } }
I20260925 02:27:21.182080 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.180202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50198 (local address 127.11.220.194:35995)
0925 02:27:21.180238 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.180240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.180246 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:27:21.180315 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.180316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.180329 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.180380 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:21.180516 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.180917 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.181559 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.181711 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.181754 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.181865 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:21.181936 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:21.181982 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.182011 (+ 29us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.182024 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:27:21.183531 16420 raft_consensus.cc:1261] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c [term 1 FOLLOWER]: Refusing update from remote peer d3d12b9401b6428abbc1b51e625e337a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:21.183535 16692 raft_consensus.cc:1261] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261 [term 1 FOLLOWER]: Refusing update from remote peer d3d12b9401b6428abbc1b51e625e337a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:21.183846 16763 consensus_queue.cc:1035] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e8017a7f634a44ac85e4ddc308b261" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:21.183955 16759 consensus_queue.cc:1035] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4a543b90f364d9984a0f3a24c07482c" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:21.185806 16766 mvcc.cc:205] Tried to move back new op lower bound from 7333082075885420544 to 7333082075824488448. Current Snapshot: MvccSnapshot[applied={T|T < 7333082075885420544}]
I20260925 02:27:21.186117 16767 mvcc.cc:205] Tried to move back new op lower bound from 7333082075885420544 to 7333082075824488448. Current Snapshot: MvccSnapshot[applied={T|T < 7333082075885420544}]
I20260925 02:27:21.188057 16274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:21.188143 16274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:21.188817 16274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 375f7662a1a847beb2a4bea73b84a45a: Sending DeleteTablet for 3 replicas of tablet 6739072b86cf40b9b4e4f3da9da91643
I20260925 02:27:21.189107 16400 tablet_service.cc:1514] Processing DeleteTablet for tablet 6739072b86cf40b9b4e4f3da9da91643 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:21 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:32870
I20260925 02:27:21.189123 16672 tablet_service.cc:1514] Processing DeleteTablet for tablet 6739072b86cf40b9b4e4f3da9da91643 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:21 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:33054
I20260925 02:27:21.189306 16534 tablet_service.cc:1514] Processing DeleteTablet for tablet 6739072b86cf40b9b4e4f3da9da91643 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:21 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:50190
I20260925 02:27:21.189419 16775 tablet_replica.cc:330] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c: stopping tablet replica
I20260925 02:27:21.189514 16775 raft_consensus.cc:2229] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:21.189527 16777 tablet_replica.cc:330] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: stopping tablet replica
I20260925 02:27:21.189574 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 16338
I20260925 02:27:21.189611 16775 raft_consensus.cc:2258] T 6739072b86cf40b9b4e4f3da9da91643 P c4a543b90f364d9984a0f3a24c07482c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:21.189718 16777 raft_consensus.cc:2229] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:21.189839 16777 raft_consensus.cc:2258] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:21.190292 16777 ts_tablet_manager.cc:1905] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:21.189749 16776 tablet_replica.cc:330] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: stopping tablet replica
I20260925 02:27:21.190526 16776 raft_consensus.cc:2229] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:21.190630 16776 raft_consensus.cc:2258] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:21.191035 16776 ts_tablet_manager.cc:1905] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:21.192257 16777 ts_tablet_manager.cc:1918] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:21.192371 16777 log.cc:1196] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/6739072b86cf40b9b4e4f3da9da91643
I20260925 02:27:21.192524 16776 ts_tablet_manager.cc:1918] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:21.192590 16776 log.cc:1196] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/6739072b86cf40b9b4e4f3da9da91643
I20260925 02:27:21.192790 16777 ts_tablet_manager.cc:1939] T 6739072b86cf40b9b4e4f3da9da91643 P d3d12b9401b6428abbc1b51e625e337a: Deleting consensus metadata
I20260925 02:27:21.192826 16776 ts_tablet_manager.cc:1939] T 6739072b86cf40b9b4e4f3da9da91643 P b8e8017a7f634a44ac85e4ddc308b261: Deleting consensus metadata
I20260925 02:27:21.193214 16259 catalog_manager.cc:4841] TS b8e8017a7f634a44ac85e4ddc308b261 (127.11.220.195:44371): tablet 6739072b86cf40b9b4e4f3da9da91643 (table test-table [id=fc75aadfd6cb451f9086fd21d2e404d2]) successfully deleted
I20260925 02:27:21.193323 16259 catalog_manager.cc:4841] TS d3d12b9401b6428abbc1b51e625e337a (127.11.220.194:35995): tablet 6739072b86cf40b9b4e4f3da9da91643 (table test-table [id=fc75aadfd6cb451f9086fd21d2e404d2]) successfully deleted
W20260925 02:27:21.197615 16261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:39321 (error 108)
I20260925 02:27:21.197778 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 16471
I20260925 02:27:21.198369 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.197894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:39321 (local address 127.0.0.1:32872)
0925 02:27:21.198302 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.198335 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:39321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392}
W20260925 02:27:21.198647 16261 catalog_manager.cc:4584] TS c4a543b90f364d9984a0f3a24c07482c (127.11.220.193:39321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6739072b86cf40b9b4e4f3da9da91643: Network error: Client connection negotiation failed: client connection to 127.11.220.193:39321: connect: Connection refused (error 111)
I20260925 02:27:21.204093 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 16609
I20260925 02:27:21.208096 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 16244
2026-09-25T02:27:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1695 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T02:27:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:21Z Disabled control of system clock
I20260925 02:27:21.233593 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:35229
--webserver_interface=127.11.220.254
--webserver_port=0
--builtin_ntp_servers=127.11.220.212:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:35229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 02:27:21.305869 16782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:21.306034 16782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:21.306052 16782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:21.307199 16782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:21.307240 16782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:21.307252 16782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:21.307264 16782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:21.307312 16782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:21.308598 16782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:35229
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:35229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:21.308816 16782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:21.309060 16782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:21.313103 16782 server_base.cc:1034] running on GCE node
W20260925 02:27:21.313040 16788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.313031 16787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.313081 16790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:21.313532 16782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:21.313792 16782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:21.314944 16782 hybrid_clock.cc:648] HybridClock initialized: now 1790303241314935 us; error 36 us; skew 500 ppm
I20260925 02:27:21.315945 16782 webserver.cc:460] Webserver started at http://127.11.220.254:34715/ using document root <none> and password file <none>
I20260925 02:27:21.316147 16782 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:21.316210 16782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:21.316344 16782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:21.317277 16782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "8b43808d4c6c4fd2857b6c197b2e83ed"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "b9b3a3b85d265b7a512b22268fec98d0"
server_key_iv: "da8a51605ff3b6eb608366cc90e2b969"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.317613 16782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "8b43808d4c6c4fd2857b6c197b2e83ed"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "b9b3a3b85d265b7a512b22268fec98d0"
server_key_iv: "da8a51605ff3b6eb608366cc90e2b969"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.318795 16782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:21.319435 16796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.319588 16782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:21.319681 16782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "8b43808d4c6c4fd2857b6c197b2e83ed"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "b9b3a3b85d265b7a512b22268fec98d0"
server_key_iv: "da8a51605ff3b6eb608366cc90e2b969"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.319753 16782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:21.322608 16782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:21.322847 16782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:21.322980 16782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:21.326426 16782 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:35229
I20260925 02:27:21.326448 16848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:35229 every 8 connection(s)
I20260925 02:27:21.326728 16782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:21.327301 16849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:21.329650 16849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed: Bootstrap starting.
I20260925 02:27:21.330142 16849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:21.330401 16849 log.cc:824] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:21.330955 16849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed: No bootstrap required, opened a new log
I20260925 02:27:21.332266 16849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b43808d4c6c4fd2857b6c197b2e83ed" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35229 } }
I20260925 02:27:21.332370 16849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:21.332410 16849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b43808d4c6c4fd2857b6c197b2e83ed, State: Initialized, Role: FOLLOWER
I20260925 02:27:21.332558 16849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b43808d4c6c4fd2857b6c197b2e83ed" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35229 } }
I20260925 02:27:21.332619 16849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:21.332638 16849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:21.332693 16849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:21.333144 16849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b43808d4c6c4fd2857b6c197b2e83ed" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35229 } }
I20260925 02:27:21.333240 16849 leader_election.cc:302] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b43808d4c6c4fd2857b6c197b2e83ed; no voters:
I20260925 02:27:21.333401 16849 leader_election.cc:288] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:21.333451 16852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:21.333590 16852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [term 1 LEADER]: Becoming Leader. State: Replica: 8b43808d4c6c4fd2857b6c197b2e83ed, State: Running, Role: LEADER
I20260925 02:27:21.333624 16849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:21.333662 16852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b43808d4c6c4fd2857b6c197b2e83ed" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35229 } }
I20260925 02:27:21.334236 16853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b43808d4c6c4fd2857b6c197b2e83ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b43808d4c6c4fd2857b6c197b2e83ed" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35229 } } }
I20260925 02:27:21.334328 16853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:21.334461 16854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b43808d4c6c4fd2857b6c197b2e83ed. Latest consensus state: current_term: 1 leader_uuid: "8b43808d4c6c4fd2857b6c197b2e83ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b43808d4c6c4fd2857b6c197b2e83ed" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35229 } } }
I20260925 02:27:21.334525 16854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:21.334547 16862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:21.334949 16862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:21.336314 16862 catalog_manager.cc:1347] Generated new cluster ID: 3f828ea9bdb24dc785f5f8f33d8788d4
I20260925 02:27:21.336360 16862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:21.336903 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 16782
I20260925 02:27:21.336990 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:21.337157 12147 external_mini_cluster.cc:1428] Setting key 93998992770c71507b01080ca5c6b2fa
I20260925 02:27:21.339020 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.337520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.11.220.254:35229)
0925 02:27:21.337685 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.337692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.337718 (+ 26us) server_negotiation.cc:407] Connection header received
0925 02:27:21.337833 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.337838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.337879 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.337987 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.338580 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.338583 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.338583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.338587 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.338618 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.338629 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.338686 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.338762 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.338881 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":83,"threads_started":1}
I20260925 02:27:21.355042 16862 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:21.355185 16862 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:21.359133 16862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed: Generated new TSK 0
I20260925 02:27:21.359284 16862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:27:21.365217 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35229
--builtin_ntp_servers=127.11.220.212:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 02:27:21.435995 16874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:21.436188 16874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:21.436220 16874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:21.437264 16874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:21.437330 16874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:21.437366 16874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:21.438483 16874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:21.438700 16874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:21.438941 16874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:21.444337 16879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.444353 16880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.444341 16882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:21.444474 16874 server_base.cc:1034] running on GCE node
I20260925 02:27:21.444664 16874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:21.444891 16874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:21.446040 16874 hybrid_clock.cc:648] HybridClock initialized: now 1790303241446025 us; error 30 us; skew 500 ppm
I20260925 02:27:21.447129 16874 webserver.cc:460] Webserver started at http://127.11.220.193:35439/ using document root <none> and password file <none>
I20260925 02:27:21.447350 16874 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:21.447388 16874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:21.447523 16874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:21.448391 16874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "d17808b9235a49dbe4e67b74625785b8"
server_key_iv: "34936995c4521e6308b9154c2cf70023"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.448757 16874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "d17808b9235a49dbe4e67b74625785b8"
server_key_iv: "34936995c4521e6308b9154c2cf70023"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.449950 16874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:21.450544 16888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.450704 16874 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:21.450771 16874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "d17808b9235a49dbe4e67b74625785b8"
server_key_iv: "34936995c4521e6308b9154c2cf70023"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.450820 16874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:21.453800 16874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:21.453990 16874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:21.454077 16874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:21.454239 16874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:21.454432 16874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:21.454458 16874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.454473 16874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:21.454484 16874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.459438 16874 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:36681
I20260925 02:27:21.459509 17001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:36681 every 8 connection(s)
I20260925 02:27:21.459774 16874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:21.462373 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.460405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:49265 (local address 127.11.220.254:35229)
0925 02:27:21.460479 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.460481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.460714 (+ 233us) server_negotiation.cc:407] Connection header received
0925 02:27:21.461139 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.461140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.461147 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.461190 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.462140 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.462141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.462141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.462146 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.462161 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.462165 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.462210 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.462298 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.462313 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 02:27:21.462532 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.460403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35229 (local address 127.11.220.193:49265)
0925 02:27:21.460609 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.460638 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.461029 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.461251 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.461254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.461261 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.462002 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:21.462003 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.462026 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:21.462049 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.462206 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.462211 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.462214 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.462307 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.462358 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":119,"threads_started":1}
I20260925 02:27:21.462947 17002 heartbeater.cc:347] Connected to a master server at 127.11.220.254:35229
I20260925 02:27:21.463024 17002 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:21.463151 17002 heartbeater.cc:511] Master 127.11.220.254:35229 requested a full tablet report, sending...
I20260925 02:27:21.463589 16813 ts_manager.cc:194] Registered new tserver with Master: 0e14724a5b3b4e0ead3fefc2131f2b2b (127.11.220.193:36681)
I20260925 02:27:21.469359 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 16874
I20260925 02:27:21.469462 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:21.469667 12147 external_mini_cluster.cc:1428] Setting key fb522293097063f1cecc515e487daf92
I20260925 02:27:21.470439 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35229
--builtin_ntp_servers=127.11.220.212:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 02:27:21.541630 17005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:21.541792 17005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:21.541811 17005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:21.542812 17005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:21.542846 17005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:21.542892 17005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:21.544188 17005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:21.544402 17005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:21.544693 17005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:21.549252 17014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.549244 17011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.549244 17012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:21.549466 17005 server_base.cc:1034] running on GCE node
I20260925 02:27:21.549719 17005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:21.549939 17005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:21.551093 17005 hybrid_clock.cc:648] HybridClock initialized: now 1790303241551074 us; error 30 us; skew 500 ppm
I20260925 02:27:21.552189 17005 webserver.cc:460] Webserver started at http://127.11.220.194:33837/ using document root <none> and password file <none>
I20260925 02:27:21.552489 17005 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:21.552551 17005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:21.552690 17005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:21.553767 17005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "9e321930d5564c3298dcedebfeaea088"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "0ced858a08e1eaaa38f2fd40fcda6882"
server_key_iv: "e3c48282ec7307e246bcfbc48bdc76d5"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.554138 17005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "9e321930d5564c3298dcedebfeaea088"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "0ced858a08e1eaaa38f2fd40fcda6882"
server_key_iv: "e3c48282ec7307e246bcfbc48bdc76d5"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.555323 17005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:21.556038 17020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.556245 17005 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:21.556349 17005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "9e321930d5564c3298dcedebfeaea088"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "0ced858a08e1eaaa38f2fd40fcda6882"
server_key_iv: "e3c48282ec7307e246bcfbc48bdc76d5"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.556435 17005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:21.559617 17005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:21.559859 17005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:21.559988 17005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:21.560204 17005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:21.560526 17005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:21.560583 17005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.560626 17005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:21.560653 17005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.566156 17005 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:32771
I20260925 02:27:21.566200 17134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:32771 every 8 connection(s)
I20260925 02:27:21.566461 17005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:21.569169 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.567041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:49713 (local address 127.11.220.254:35229)
0925 02:27:21.567100 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.567103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.567299 (+ 196us) server_negotiation.cc:407] Connection header received
0925 02:27:21.567729 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.567730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.567738 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.567802 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.568907 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.568909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.568909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.568914 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.568932 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.568937 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.568992 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.569083 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.569099 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:27:21.569360 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.567040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35229 (local address 127.11.220.194:49713)
0925 02:27:21.567197 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.567226 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.567630 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.567870 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.567873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.567880 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.568767 (+ 887us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:21.568767 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.568792 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:21.568817 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.568996 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.569001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.569005 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.569092 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.569150 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260925 02:27:21.569864 17135 heartbeater.cc:347] Connected to a master server at 127.11.220.254:35229
I20260925 02:27:21.569959 17135 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:21.570115 17135 heartbeater.cc:511] Master 127.11.220.254:35229 requested a full tablet report, sending...
I20260925 02:27:21.570433 16813 ts_manager.cc:194] Registered new tserver with Master: 9e321930d5564c3298dcedebfeaea088 (127.11.220.194:32771)
I20260925 02:27:21.574281 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 17005
I20260925 02:27:21.574361 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:21.574491 12147 external_mini_cluster.cc:1428] Setting key 26c7afa022cbc08012d8d76ad6f042a8
I20260925 02:27:21.575273 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35229
--builtin_ntp_servers=127.11.220.212:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 02:27:21.649067 17138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:21.649226 17138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:21.649246 17138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:21.650213 17138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:21.650238 17138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:21.650259 17138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:21.651360 17138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:21.651510 17138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:21.651717 17138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:21.656344 17143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.656353 17144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:21.656383 17138 server_base.cc:1034] running on GCE node
W20260925 02:27:21.656354 17146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:21.656780 17138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:21.656975 17138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:21.658119 17138 hybrid_clock.cc:648] HybridClock initialized: now 1790303241658100 us; error 36 us; skew 500 ppm
I20260925 02:27:21.659277 17138 webserver.cc:460] Webserver started at http://127.11.220.195:36989/ using document root <none> and password file <none>
I20260925 02:27:21.659489 17138 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:21.659530 17138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:21.659667 17138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:21.660650 17138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "2e8dc7020bc143f9a2c914cb21ae0724"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "10546b20a36966ca77537805063ba1b5"
server_key_iv: "0908926b34c509772317abfe2994c360"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.660956 17138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "2e8dc7020bc143f9a2c914cb21ae0724"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "10546b20a36966ca77537805063ba1b5"
server_key_iv: "0908926b34c509772317abfe2994c360"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.662120 17138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:21.662716 17153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.662860 17138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:21.662947 17138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "2e8dc7020bc143f9a2c914cb21ae0724"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "10546b20a36966ca77537805063ba1b5"
server_key_iv: "0908926b34c509772317abfe2994c360"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.663019 17138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:21.665905 17138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:21.666126 17138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:21.666254 17138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:21.666432 17138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:21.666698 17138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:21.666745 17138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.666779 17138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:21.666803 17138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.671742 17138 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:33921
I20260925 02:27:21.671806 17268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:33921 every 8 connection(s)
I20260925 02:27:21.672073 17138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:21.674467 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.672685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:50723 (local address 127.11.220.254:35229)
0925 02:27:21.672737 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.672739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.672937 (+ 198us) server_negotiation.cc:407] Connection header received
0925 02:27:21.673286 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.673287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.673291 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.673335 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.674316 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.674317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.674317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.674319 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.674328 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.674331 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.674366 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.674414 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.674424 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 02:27:21.674634 17271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.672665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35229 (local address 127.11.220.195:50723)
0925 02:27:21.672830 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.672862 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.673217 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.673382 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.673385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.673391 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.674197 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:21.674198 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.674219 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:21.674242 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.674365 (+ 123us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.674367 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.674369 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.674409 (+ 40us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.674447 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260925 02:27:21.675091 17269 heartbeater.cc:347] Connected to a master server at 127.11.220.254:35229
I20260925 02:27:21.675181 17269 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:21.675311 17269 heartbeater.cc:511] Master 127.11.220.254:35229 requested a full tablet report, sending...
I20260925 02:27:21.675602 16813 ts_manager.cc:194] Registered new tserver with Master: 2e8dc7020bc143f9a2c914cb21ae0724 (127.11.220.195:33921)
I20260925 02:27:21.679294 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 17138
I20260925 02:27:21.679369 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:21.679482 12147 external_mini_cluster.cc:1428] Setting key 3a7e410a89434ce05d79522f2c118b9f
I20260925 02:27:21.680210 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:27:21.682364 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.681404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51004 (local address 127.11.220.254:35229)
0925 02:27:21.681471 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.681472 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.681564 (+ 92us) server_negotiation.cc:407] Connection header received
0925 02:27:21.681617 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.681618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.681622 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.681668 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.682193 (+ 525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.682194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.682194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.682198 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.682211 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.682214 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.682257 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.682296 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.682310 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:27:21.683585 16813 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:21.685081 16813 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:21.688485 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.686969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:32771 (local address 127.0.0.1:35598)
0925 02:27:21.687324 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.687358 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.687422 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.687552 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.687556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.687561 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.688027 (+ 466us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:21.688028 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.688041 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:21.688060 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.688300 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.688307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.688310 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.688374 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.688385 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":86,"threads_started":1}
I20260925 02:27:21.688534 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.687125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35598 (local address 127.11.220.194:32771)
0925 02:27:21.687367 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.687372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.687427 (+ 55us) server_negotiation.cc:407] Connection header received
0925 02:27:21.687465 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.687466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.687474 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.687530 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.688148 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.688155 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.688155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.688160 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.688192 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.688231 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.688305 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.688380 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.688396 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":75,"threads_started":1}
I20260925 02:27:21.688999 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.687140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47638 (local address 127.11.220.193:36681)
0925 02:27:21.687290 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.687294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.687606 (+ 312us) server_negotiation.cc:407] Connection header received
0925 02:27:21.687637 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.687640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.687649 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.687715 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.688686 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.688696 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.688697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.688701 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.688737 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.688764 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.688797 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.688909 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.688924 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260925 02:27:21.689037 17278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.686974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:33921 (local address 127.0.0.1:43606)
0925 02:27:21.687634 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.687643 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.687679 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.688308 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.688309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.688311 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.688716 (+ 405us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:21.688717 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.688724 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:21.688735 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.688923 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.688924 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.688924 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.688962 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.688973 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":88,"threads_started":1}
I20260925 02:27:21.689100 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.687178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.11.220.195:33921)
0925 02:27:21.687432 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.687435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.687946 (+ 511us) server_negotiation.cc:407] Connection header received
0925 02:27:21.687970 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.687972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.687980 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.688029 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.688811 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.688817 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.688818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.688822 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.688851 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.688869 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.688915 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.688976 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.688988 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":59,"threads_started":1}
I20260925 02:27:21.689143 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.686969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:36681 (local address 127.0.0.1:47638)
0925 02:27:21.687500 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.687506 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.687561 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.687962 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.687963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.687965 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.688582 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:21.688583 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.688589 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:21.688600 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.688842 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.688843 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.688843 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.689077 (+ 234us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.689090 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":4,"thread_start_us":49,"threads_started":1}
I20260925 02:27:21.689208 17068 tablet_service.cc:1467] Processing CreateTablet for tablet be56945303e8409989cca49a6b2e6bd9 (DEFAULT_TABLE table=test-table [id=0d6b976a3f10470faddb4d90c392c0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:21.689488 17068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be56945303e8409989cca49a6b2e6bd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:21.689575 17201 tablet_service.cc:1467] Processing CreateTablet for tablet be56945303e8409989cca49a6b2e6bd9 (DEFAULT_TABLE table=test-table [id=0d6b976a3f10470faddb4d90c392c0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:21.689857 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be56945303e8409989cca49a6b2e6bd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:21.691751 17283 tablet_bootstrap.cc:492] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: Bootstrap starting.
I20260925 02:27:21.692286 16936 tablet_service.cc:1467] Processing CreateTablet for tablet be56945303e8409989cca49a6b2e6bd9 (DEFAULT_TABLE table=test-table [id=0d6b976a3f10470faddb4d90c392c0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:21.692417 17283 tablet_bootstrap.cc:654] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:21.692628 16936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be56945303e8409989cca49a6b2e6bd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:21.692690 17284 tablet_bootstrap.cc:492] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: Bootstrap starting.
I20260925 02:27:21.692792 17283 log.cc:824] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:21.693339 17284 tablet_bootstrap.cc:654] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:21.693570 17283 tablet_bootstrap.cc:492] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: No bootstrap required, opened a new log
I20260925 02:27:21.693638 17283 ts_tablet_manager.cc:1397] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 02:27:21.693631 17284 log.cc:824] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:21.694221 17284 tablet_bootstrap.cc:492] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: No bootstrap required, opened a new log
I20260925 02:27:21.694298 17284 ts_tablet_manager.cc:1397] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:21.695689 17287 tablet_bootstrap.cc:492] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b: Bootstrap starting.
I20260925 02:27:21.695993 17283 raft_consensus.cc:348] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } }
I20260925 02:27:21.696131 17283 raft_consensus.cc:374] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:21.696170 17283 raft_consensus.cc:729] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e321930d5564c3298dcedebfeaea088, State: Initialized, Role: FOLLOWER
I20260925 02:27:21.696287 17283 consensus_queue.cc:260] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } }
I20260925 02:27:21.696390 17287 tablet_bootstrap.cc:654] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:21.696348 17284 raft_consensus.cc:348] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } }
I20260925 02:27:21.696527 17284 raft_consensus.cc:374] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:21.696548 17283 ts_tablet_manager.cc:1428] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 02:27:21.696575 17284 raft_consensus.cc:729] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8dc7020bc143f9a2c914cb21ae0724, State: Initialized, Role: FOLLOWER
I20260925 02:27:21.696650 17135 heartbeater.cc:503] Master 127.11.220.254:35229 was elected leader, sending a full tablet report...
I20260925 02:27:21.696702 17287 log.cc:824] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:21.696686 17284 consensus_queue.cc:260] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } }
I20260925 02:27:21.696911 17284 ts_tablet_manager.cc:1428] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:27:21.697001 17269 heartbeater.cc:503] Master 127.11.220.254:35229 was elected leader, sending a full tablet report...
I20260925 02:27:21.697456 17287 tablet_bootstrap.cc:492] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b: No bootstrap required, opened a new log
I20260925 02:27:21.697530 17287 ts_tablet_manager.cc:1397] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:21.699283 17287 raft_consensus.cc:348] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } }
I20260925 02:27:21.699388 17287 raft_consensus.cc:374] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:21.699406 17287 raft_consensus.cc:729] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e14724a5b3b4e0ead3fefc2131f2b2b, State: Initialized, Role: FOLLOWER
I20260925 02:27:21.699524 17287 consensus_queue.cc:260] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } }
I20260925 02:27:21.699792 17287 ts_tablet_manager.cc:1428] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:27:21.699811 17002 heartbeater.cc:503] Master 127.11.220.254:35229 was elected leader, sending a full tablet report...
W20260925 02:27:21.710233 17003 tablet.cc:2369] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:21.746508 17288 raft_consensus.cc:482] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:21.746579 17288 raft_consensus.cc:504] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } }
I20260925 02:27:21.746838 17288 leader_election.cc:288] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e14724a5b3b4e0ead3fefc2131f2b2b (127.11.220.193:36681), 2e8dc7020bc143f9a2c914cb21ae0724 (127.11.220.195:33921)
I20260925 02:27:21.746958 17289 raft_consensus.cc:482] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:21.747045 17289 raft_consensus.cc:504] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } }
I20260925 02:27:21.747378 17289 leader_election.cc:288] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e321930d5564c3298dcedebfeaea088 (127.11.220.194:32771), 0e14724a5b3b4e0ead3fefc2131f2b2b (127.11.220.193:36681)
I20260925 02:27:21.747960 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.746963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:36681 (local address 127.11.220.194:40233)
0925 02:27:21.747019 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.747030 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.747064 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.747204 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.747206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.747208 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.747643 (+ 435us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:21.747644 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.747652 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:21.747666 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.747825 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.747826 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.747826 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.747864 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.747876 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 02:27:21.748178 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.747041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:40233 (local address 127.11.220.193:36681)
0925 02:27:21.747081 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.747083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.747106 (+ 23us) server_negotiation.cc:407] Connection header received
0925 02:27:21.747134 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.747135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.747140 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.747184 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.747737 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.747738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.747739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.747742 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.747755 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.747760 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.747802 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.748109 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.748121 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:27:21.748418 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be56945303e8409989cca49a6b2e6bd9" candidate_uuid: "9e321930d5564c3298dcedebfeaea088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" is_pre_election: true
I20260925 02:27:21.748550 16956 raft_consensus.cc:2454] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e321930d5564c3298dcedebfeaea088 in term 0.
I20260925 02:27:21.748672 17271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.747450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:32771 (local address 127.11.220.195:35525)
0925 02:27:21.747510 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.747518 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.747546 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.747697 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.747698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.747699 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.748158 (+ 459us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:21.748158 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.748164 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:21.748174 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.748547 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.748548 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.748548 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.748591 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.748601 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 02:27:21.748693 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.747455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:36681 (local address 127.11.220.195:37619)
0925 02:27:21.747788 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.747796 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.747829 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.748001 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.748002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.748004 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.748382 (+ 378us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:21.748382 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.748388 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:21.748401 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.748585 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.748585 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.748586 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.748622 (+ 36us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.748631 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":165,"thread_start_us":73,"threads_started":1}
I20260925 02:27:21.748751 17021 leader_election.cc:302] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e14724a5b3b4e0ead3fefc2131f2b2b, 9e321930d5564c3298dcedebfeaea088; no voters:
I20260925 02:27:21.748862 17288 raft_consensus.cc:2781] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:21.748917 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.747523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:35525 (local address 127.11.220.194:32771)
0925 02:27:21.747567 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.747569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.747577 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:21.747594 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.747594 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.747598 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.747649 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.748247 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.748248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.748248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.748252 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.748264 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.748268 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.748328 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.748751 (+ 423us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.748764 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:27:21.748950 17288 raft_consensus.cc:482] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:21.748974 17288 raft_consensus.cc:3037] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:21.749097 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.746974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:33921 (local address 127.11.220.194:35115)
0925 02:27:21.747151 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.747159 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.747189 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.747714 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.747714 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.747716 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:21.748438 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:21.748438 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:21.748443 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:21.748453 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:21.748917 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:21.748918 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:21.748923 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:27:21.749023 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 02:27:21.749035 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260925 02:27:21.749121 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.747239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:35115 (local address 127.11.220.195:33921)
0925 02:27:21.747295 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.747296 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.747303 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:21.747607 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.747608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.747613 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.747656 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.748521 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.748522 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.748522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.748525 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.748536 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.748539 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.748591 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.749043 (+ 452us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.749057 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 02:27:21.749290 17088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be56945303e8409989cca49a6b2e6bd9" candidate_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e321930d5564c3298dcedebfeaea088" is_pre_election: true
I20260925 02:27:21.749467 17222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be56945303e8409989cca49a6b2e6bd9" candidate_uuid: "9e321930d5564c3298dcedebfeaea088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" is_pre_election: true
I20260925 02:27:21.749533 17222 raft_consensus.cc:2454] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e321930d5564c3298dcedebfeaea088 in term 0.
I20260925 02:27:21.750010 17288 raft_consensus.cc:504] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } }
I20260925 02:27:21.750144 17088 raft_consensus.cc:2379] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2e8dc7020bc143f9a2c914cb21ae0724 in current term 1: Already voted for candidate 9e321930d5564c3298dcedebfeaea088 in this term.
I20260925 02:27:21.750254 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.747520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:37619 (local address 127.11.220.193:36681)
0925 02:27:21.747863 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.747865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.747872 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:21.747889 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.747890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.747894 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.747950 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.748497 (+ 547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.748498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.748498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.748502 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.748522 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.748526 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.748566 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.748806 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.748819 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":73,"threads_started":1}
I20260925 02:27:21.750151 17288 leader_election.cc:288] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [CANDIDATE]: Term 1 election: Requested vote from peers 0e14724a5b3b4e0ead3fefc2131f2b2b (127.11.220.193:36681), 2e8dc7020bc143f9a2c914cb21ae0724 (127.11.220.195:33921)
I20260925 02:27:21.750267 17222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be56945303e8409989cca49a6b2e6bd9" candidate_uuid: "9e321930d5564c3298dcedebfeaea088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8dc7020bc143f9a2c914cb21ae0724"
I20260925 02:27:21.750267 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be56945303e8409989cca49a6b2e6bd9" candidate_uuid: "9e321930d5564c3298dcedebfeaea088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b"
I20260925 02:27:21.750327 17222 raft_consensus.cc:3037] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:21.750327 16956 raft_consensus.cc:3037] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:21.750481 16955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be56945303e8409989cca49a6b2e6bd9" candidate_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" is_pre_election: true
I20260925 02:27:21.750789 17155 leader_election.cc:302] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2e8dc7020bc143f9a2c914cb21ae0724; no voters: 0e14724a5b3b4e0ead3fefc2131f2b2b, 9e321930d5564c3298dcedebfeaea088
I20260925 02:27:21.751080 17222 raft_consensus.cc:2454] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e321930d5564c3298dcedebfeaea088 in term 1.
I20260925 02:27:21.751076 16956 raft_consensus.cc:2454] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e321930d5564c3298dcedebfeaea088 in term 1.
I20260925 02:27:21.751212 17289 raft_consensus.cc:2726] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 02:27:21.751235 17021 leader_election.cc:302] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e14724a5b3b4e0ead3fefc2131f2b2b, 9e321930d5564c3298dcedebfeaea088; no voters:
I20260925 02:27:21.751307 17288 raft_consensus.cc:2781] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:21.751441 17288 raft_consensus.cc:686] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 1 LEADER]: Becoming Leader. State: Replica: 9e321930d5564c3298dcedebfeaea088, State: Running, Role: LEADER
I20260925 02:27:21.751534 17288 consensus_queue.cc:237] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } }
I20260925 02:27:21.752274 16813 catalog_manager.cc:5495] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e321930d5564c3298dcedebfeaea088 (127.11.220.194). New cstate: current_term: 1 leader_uuid: "9e321930d5564c3298dcedebfeaea088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e321930d5564c3298dcedebfeaea088" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 32771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 } health_report { overall_health: UNKNOWN } } }
I20260925 02:27:21.761212 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.760413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35614 (local address 127.11.220.194:32771)
0925 02:27:21.760470 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.760472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.760479 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:21.760553 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.760554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.760564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.760634 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.761031 (+ 397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.761031 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.761032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.761035 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.761045 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.761048 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.761095 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.761144 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.761157 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:21.762564 16956 raft_consensus.cc:1261] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b [term 1 FOLLOWER]: Refusing update from remote peer 9e321930d5564c3298dcedebfeaea088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:21.762574 17222 raft_consensus.cc:1261] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 1 FOLLOWER]: Refusing update from remote peer 9e321930d5564c3298dcedebfeaea088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:21.762848 17288 consensus_queue.cc:1035] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8dc7020bc143f9a2c914cb21ae0724" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 33921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:21.762957 17295 consensus_queue.cc:1035] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e14724a5b3b4e0ead3fefc2131f2b2b" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:21.765408 17299 mvcc.cc:205] Tried to move back new op lower bound from 7333082078257475584 to 7333082078215032832. Current Snapshot: MvccSnapshot[applied={T|T < 7333082078257475584}]
I20260925 02:27:21.765717 17300 mvcc.cc:205] Tried to move back new op lower bound from 7333082078257475584 to 7333082078215032832. Current Snapshot: MvccSnapshot[applied={T|T < 7333082078257475584}]
I20260925 02:27:21.765944 17298 mvcc.cc:205] Tried to move back new op lower bound from 7333082078257475584 to 7333082078215032832. Current Snapshot: MvccSnapshot[applied={T|T < 7333082078257475584 or (T in {7333082078257475584})}]
I20260925 02:27:21.766943 16812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:21.767028 16812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:21.767653 16812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b43808d4c6c4fd2857b6c197b2e83ed: Sending DeleteTablet for 3 replicas of tablet be56945303e8409989cca49a6b2e6bd9
I20260925 02:27:21.767992 16936 tablet_service.cc:1514] Processing DeleteTablet for tablet be56945303e8409989cca49a6b2e6bd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:21 UTC) from {username='slave'} at 127.0.0.1:47638
I20260925 02:27:21.767979 17201 tablet_service.cc:1514] Processing DeleteTablet for tablet be56945303e8409989cca49a6b2e6bd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:21 UTC) from {username='slave'} at 127.0.0.1:43606
I20260925 02:27:21.768080 17068 tablet_service.cc:1514] Processing DeleteTablet for tablet be56945303e8409989cca49a6b2e6bd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:21 UTC) from {username='slave'} at 127.0.0.1:35598
I20260925 02:27:21.768195 17308 tablet_replica.cc:330] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b: stopping tablet replica
I20260925 02:27:21.768287 17308 raft_consensus.cc:2229] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:21.768328 17309 tablet_replica.cc:330] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: stopping tablet replica
I20260925 02:27:21.768394 17308 raft_consensus.cc:2258] T be56945303e8409989cca49a6b2e6bd9 P 0e14724a5b3b4e0ead3fefc2131f2b2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:21.768515 17309 raft_consensus.cc:2229] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:21.768646 17309 raft_consensus.cc:2258] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:21.768806 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 16874
I20260925 02:27:21.770969 17309 ts_tablet_manager.cc:1905] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:21.772540 17310 tablet_replica.cc:330] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: stopping tablet replica
I20260925 02:27:21.772553 17309 ts_tablet_manager.cc:1918] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:21.772643 17310 raft_consensus.cc:2229] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:21.772737 17309 log.cc:1196] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/be56945303e8409989cca49a6b2e6bd9
I20260925 02:27:21.772749 17310 raft_consensus.cc:2258] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:21.773059 17309 ts_tablet_manager.cc:1939] T be56945303e8409989cca49a6b2e6bd9 P 2e8dc7020bc143f9a2c914cb21ae0724: Deleting consensus metadata
I20260925 02:27:21.773130 17310 ts_tablet_manager.cc:1905] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:21.773470 16799 catalog_manager.cc:4841] TS 2e8dc7020bc143f9a2c914cb21ae0724 (127.11.220.195:33921): tablet be56945303e8409989cca49a6b2e6bd9 (table test-table [id=0d6b976a3f10470faddb4d90c392c0b7]) successfully deleted
I20260925 02:27:21.774492 17310 ts_tablet_manager.cc:1918] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:21.774556 17310 log.cc:1196] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/be56945303e8409989cca49a6b2e6bd9
I20260925 02:27:21.774788 17310 ts_tablet_manager.cc:1939] T be56945303e8409989cca49a6b2e6bd9 P 9e321930d5564c3298dcedebfeaea088: Deleting consensus metadata
I20260925 02:27:21.775106 16798 catalog_manager.cc:4841] TS 9e321930d5564c3298dcedebfeaea088 (127.11.220.194:32771): tablet be56945303e8409989cca49a6b2e6bd9 (table test-table [id=0d6b976a3f10470faddb4d90c392c0b7]) successfully deleted
W20260925 02:27:21.775154 16797 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:36681 (error 108)
I20260925 02:27:21.775600 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.775485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:36681 (local address 127.0.0.1:47646)
0925 02:27:21.775536 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.775570 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 02:27:21.775661 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 17005
W20260925 02:27:21.777273 16797 catalog_manager.cc:4584] TS 0e14724a5b3b4e0ead3fefc2131f2b2b (127.11.220.193:36681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be56945303e8409989cca49a6b2e6bd9: Network error: Client connection negotiation failed: client connection to 127.11.220.193:36681: connect: Connection refused (error 111)
I20260925 02:27:21.780792 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 17138
I20260925 02:27:21.785055 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 16782
2026-09-25T02:27:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (576 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 02:27:21.803637 12147 mini_oidc.cc:152] Starting JWKS server
I20260925 02:27:21.804669 12147 webserver.cc:460] Webserver started at https://127.0.0.1:33655/ using document root <none> and password file <none>
I20260925 02:27:21.804739 12147 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 02:27:21.804911 12147 webserver.cc:460] Webserver started at http://127.0.0.1:41183/ using document root <none> and password file <none>
2026-09-25T02:27:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:21Z Disabled control of system clock
I20260925 02:27:21.811264 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:42979
--webserver_interface=127.11.220.254
--webserver_port=0
--builtin_ntp_servers=127.11.220.212:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:42979
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33655/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/testchainca.cert with env {}
W20260925 02:27:21.881799 17317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:21.881953 17317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:21.881971 17317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:21.883359 17317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:21.883395 17317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:21.883406 17317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:21.883417 17317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:21.883428 17317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 02:27:21.883440 17317 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 02:27:21.883457 17317 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33655/jwks.json
I20260925 02:27:21.884752 17317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:42979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:42979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33655/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:21.884905 17317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:21.885038 17317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:21.889295 17325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.889294 17322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:21.889348 17323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:21.889596 17317 server_base.cc:1034] running on GCE node
I20260925 02:27:21.889789 17317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:21.889985 17317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:21.891137 17317 hybrid_clock.cc:648] HybridClock initialized: now 1790303241891131 us; error 33 us; skew 500 ppm
I20260925 02:27:21.892274 17317 webserver.cc:460] Webserver started at http://127.11.220.254:38825/ using document root <none> and password file <none>
I20260925 02:27:21.892534 17317 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:21.892597 17317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:21.892735 17317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:21.893697 17317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "6f9439b616fe4ce6b337c1cc4d8b0961"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "6feb6f9df4b9c876921f34a3afcbbd76"
server_key_iv: "6eba90361fb074bce761a891bfa73d76"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.894018 17317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "6f9439b616fe4ce6b337c1cc4d8b0961"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "6feb6f9df4b9c876921f34a3afcbbd76"
server_key_iv: "6eba90361fb074bce761a891bfa73d76"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.895197 17317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:21.895819 17331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:21.895979 17317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:21.896067 17317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "6f9439b616fe4ce6b337c1cc4d8b0961"
format_stamp: "Formatted at 2026-09-25 02:27:21 on dist-test-slave-dn4n"
server_key: "6feb6f9df4b9c876921f34a3afcbbd76"
server_key_iv: "6eba90361fb074bce761a891bfa73d76"
server_key_version: "encryptionkey@0"
I20260925 02:27:21.896143 17317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:21.910907 17317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:21.911144 17317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:21.911278 17317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:21.914585 17317 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:42979
I20260925 02:27:21.914649 17383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:42979 every 8 connection(s)
I20260925 02:27:21.914908 17317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:21.915683 17384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:21.917663 17384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961: Bootstrap starting.
I20260925 02:27:21.918215 17384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:21.918481 17384 log.cc:824] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:21.919034 17384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961: No bootstrap required, opened a new log
I20260925 02:27:21.920545 17384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9439b616fe4ce6b337c1cc4d8b0961" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42979 } }
I20260925 02:27:21.920661 17384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:21.920681 17384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9439b616fe4ce6b337c1cc4d8b0961, State: Initialized, Role: FOLLOWER
I20260925 02:27:21.920826 17384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9439b616fe4ce6b337c1cc4d8b0961" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42979 } }
I20260925 02:27:21.920879 17384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:21.920917 17384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:21.920960 17384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:21.921408 17384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9439b616fe4ce6b337c1cc4d8b0961" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42979 } }
I20260925 02:27:21.921504 17384 leader_election.cc:302] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f9439b616fe4ce6b337c1cc4d8b0961; no voters:
I20260925 02:27:21.921660 17384 leader_election.cc:288] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:21.921717 17387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:21.921856 17387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [term 1 LEADER]: Becoming Leader. State: Replica: 6f9439b616fe4ce6b337c1cc4d8b0961, State: Running, Role: LEADER
I20260925 02:27:21.921875 17384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:21.921947 17387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9439b616fe4ce6b337c1cc4d8b0961" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42979 } }
I20260925 02:27:21.922348 17388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f9439b616fe4ce6b337c1cc4d8b0961" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9439b616fe4ce6b337c1cc4d8b0961" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42979 } } }
I20260925 02:27:21.922423 17388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:21.922573 17389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f9439b616fe4ce6b337c1cc4d8b0961. Latest consensus state: current_term: 1 leader_uuid: "6f9439b616fe4ce6b337c1cc4d8b0961" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9439b616fe4ce6b337c1cc4d8b0961" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42979 } } }
I20260925 02:27:21.922603 17397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:21.922631 17389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:21.923049 17397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:21.924592 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 17317
I20260925 02:27:21.924413 17397 catalog_manager.cc:1347] Generated new cluster ID: ff4d22ac688948f481b7efdb06854176
I20260925 02:27:21.924649 17397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:21.924690 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:21.924860 12147 external_mini_cluster.cc:1428] Setting key 45c145b7de93e25cb8351e8985e1975c
I20260925 02:27:21.930114 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.925216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49850 (local address 127.11.220.254:42979)
0925 02:27:21.925389 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.925397 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.925443 (+ 46us) server_negotiation.cc:407] Connection header received
0925 02:27:21.925583 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.925588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.925647 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.925752 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:21.925950 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.927095 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.928160 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.928788 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.928898 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.929514 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:21.929517 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:21.929517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:21.929690 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:21.929722 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:21.929734 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:21.929785 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:21.929871 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 02:27:21.929930 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260925 02:27:21.930958 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:27:21.948869 17397 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:21.949304 17397 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:21.951377 17397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f9439b616fe4ce6b337c1cc4d8b0961: Generated new TSK 0
I20260925 02:27:21.951531 17397 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 02:27:21.976996 17415 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 02:27:21.977083 17415 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260925 02:27:21.977615 17414 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:21.963336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:42979 (local address 127.0.0.1:49858)
0925 02:27:21.963528 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:21.963537 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:21.963586 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:21.963747 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:21.963748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:21.963749 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0925 02:27:21.963808 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.963809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.964706 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.964707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.965227 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:21.965228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.965639 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.965640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:21.965685 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.965702 (+ 17us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 02:27:21.977227 (+ 11525us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33655/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 02:27:21.977316 (+ 89us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.220.254:42979: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33655/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
I20260925 02:27:21.977885 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:21.963417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.11.220.254:42979)
0925 02:27:21.963473 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:21.963475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:21.963588 (+ 113us) server_negotiation.cc:407] Connection header received
0925 02:27:21.963652 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:21.963653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:21.963671 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:21.963747 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0925 02:27:21.963865 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.964618 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.965297 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:21.965570 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:21.965635 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:21.965794 (+ 159us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 02:27:21.977106 (+ 11312us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33655/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 02:27:21.977831 (+ 725us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49858: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33655/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260925 02:27:21.977926 17408 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49858: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33655/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.220.254:42979: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33655/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 02:27:21.978334 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 17317
2026-09-25T02:27:21Z chronyd exiting
I20260925 02:27:22.014132 12147 test_util.cc:175] -----------------------------------------------
I20260925 02:27:22.014181 12147 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790303222243611-12147-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (211 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 02:27:22.014775 12147 mini_oidc.cc:152] Starting JWKS server
I20260925 02:27:22.015329 12147 webserver.cc:460] Webserver started at https://127.0.0.1:45927/ using document root <none> and password file <none>
I20260925 02:27:22.015363 12147 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 02:27:22.015487 12147 webserver.cc:460] Webserver started at http://127.0.0.1:36005/ using document root <none> and password file <none>
2026-09-25T02:27:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:22Z Disabled control of system clock
I20260925 02:27:22.022333 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43099
--webserver_interface=127.11.220.254
--webserver_port=0
--builtin_ntp_servers=127.11.220.212:36909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:43099
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45927/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/testchainca.cert with env {}
W20260925 02:27:22.094676 17421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:22.094851 17421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:22.094870 17421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:22.095906 17421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:22.095961 17421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:22.095973 17421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:22.095985 17421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:22.095996 17421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 02:27:22.096009 17421 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 02:27:22.096032 17421 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45927/jwks.json
I20260925 02:27:22.097379 17421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:36909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:43099
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45927/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:22.097543 17421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:22.097681 17421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:22.101863 17421 server_base.cc:1034] running on GCE node
W20260925 02:27:22.101795 17427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:22.101795 17430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:22.101811 17426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:22.102173 17421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:22.102387 17421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:22.103544 17421 hybrid_clock.cc:648] HybridClock initialized: now 1790303242103532 us; error 26 us; skew 500 ppm
I20260925 02:27:22.104605 17421 webserver.cc:460] Webserver started at http://127.11.220.254:33035/ using document root <none> and password file <none>
I20260925 02:27:22.104820 17421 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:22.104880 17421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:22.105000 17421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:22.105957 17421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "afa184984f3a41e9bebcb45a0fd572c7"
format_stamp: "Formatted at 2026-09-25 02:27:22 on dist-test-slave-dn4n"
server_key: "fdbbe8483239f8504dd70b74cb834d56"
server_key_iv: "b6cd291469b59f9ae1887ebe739b3f0e"
server_key_version: "encryptionkey@0"
I20260925 02:27:22.106297 17421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "afa184984f3a41e9bebcb45a0fd572c7"
format_stamp: "Formatted at 2026-09-25 02:27:22 on dist-test-slave-dn4n"
server_key: "fdbbe8483239f8504dd70b74cb834d56"
server_key_iv: "b6cd291469b59f9ae1887ebe739b3f0e"
server_key_version: "encryptionkey@0"
I20260925 02:27:22.107470 17421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:22.108112 17436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:22.108265 17421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:22.108351 17421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "afa184984f3a41e9bebcb45a0fd572c7"
format_stamp: "Formatted at 2026-09-25 02:27:22 on dist-test-slave-dn4n"
server_key: "fdbbe8483239f8504dd70b74cb834d56"
server_key_iv: "b6cd291469b59f9ae1887ebe739b3f0e"
server_key_version: "encryptionkey@0"
I20260925 02:27:22.108428 17421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:22.131225 17421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:22.131481 17421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:22.131603 17421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:22.135277 17421 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:43099
I20260925 02:27:22.135303 17488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:43099 every 8 connection(s)
I20260925 02:27:22.135576 17421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:22.135917 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 17421
I20260925 02:27:22.136018 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:22.136211 12147 external_mini_cluster.cc:1428] Setting key d791c2621813d27a67fd215ee1a9677c
I20260925 02:27:22.136258 17489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:22.139153 17489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7: Bootstrap starting.
I20260925 02:27:22.139689 17489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:22.139954 17489 log.cc:824] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:22.140640 17489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7: No bootstrap required, opened a new log
I20260925 02:27:22.141779 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:22.136841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43568 (local address 127.11.220.254:43099)
0925 02:27:22.137022 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:22.137029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:22.137056 (+ 27us) server_negotiation.cc:407] Connection header received
0925 02:27:22.137528 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:22.137536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:22.137608 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:22.137741 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:22.137888 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.139139 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.139871 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.140396 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.140527 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:22.141139 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:22.141144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:22.141144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:22.141329 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:22.141367 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:22.141382 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:22.141452 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:22.141507 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 02:27:22.141577 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260925 02:27:22.142774 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:27:22.142969 17489 raft_consensus.cc:348] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa184984f3a41e9bebcb45a0fd572c7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43099 } }
I20260925 02:27:22.143074 17489 raft_consensus.cc:374] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:22.143093 17489 raft_consensus.cc:729] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa184984f3a41e9bebcb45a0fd572c7, State: Initialized, Role: FOLLOWER
I20260925 02:27:22.143231 17489 consensus_queue.cc:260] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa184984f3a41e9bebcb45a0fd572c7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43099 } }
I20260925 02:27:22.143296 17489 raft_consensus.cc:388] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:22.143316 17489 raft_consensus.cc:482] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:22.143373 17489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:22.143949 17489 raft_consensus.cc:504] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa184984f3a41e9bebcb45a0fd572c7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43099 } }
I20260925 02:27:22.144026 17489 leader_election.cc:302] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afa184984f3a41e9bebcb45a0fd572c7; no voters:
I20260925 02:27:22.144129 17489 leader_election.cc:288] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:22.144203 17494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:22.144331 17494 raft_consensus.cc:686] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [term 1 LEADER]: Becoming Leader. State: Replica: afa184984f3a41e9bebcb45a0fd572c7, State: Running, Role: LEADER
I20260925 02:27:22.144344 17489 sys_catalog.cc:564] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:22.144403 17494 consensus_queue.cc:237] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa184984f3a41e9bebcb45a0fd572c7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43099 } }
I20260925 02:27:22.144683 17495 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afa184984f3a41e9bebcb45a0fd572c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa184984f3a41e9bebcb45a0fd572c7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43099 } } }
I20260925 02:27:22.144769 17495 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:22.144683 17496 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afa184984f3a41e9bebcb45a0fd572c7. Latest consensus state: current_term: 1 leader_uuid: "afa184984f3a41e9bebcb45a0fd572c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa184984f3a41e9bebcb45a0fd572c7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43099 } } }
I20260925 02:27:22.144945 17496 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:22.144959 17501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:22.145560 17501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:22.146840 17501 catalog_manager.cc:1347] Generated new cluster ID: bf68d3be2ae3422ab177be7baebf74d6
I20260925 02:27:22.146883 17501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:22.155464 17501 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:22.155887 17501 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:22.165249 17501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afa184984f3a41e9bebcb45a0fd572c7: Generated new TSK 0
I20260925 02:27:22.165391 17501 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 02:27:22.192011 17520 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 02:27:22.192060 17520 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 02:27:22.192301 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:22.172492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43574 (local address 127.11.220.254:43099)
0925 02:27:22.172529 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:22.172531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:22.172586 (+ 55us) server_negotiation.cc:407] Connection header received
0925 02:27:22.172648 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:22.172648 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:22.172656 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:22.172701 (+ 45us) server_negotiation.cc:226] Negotiated authn=JWT
0925 02:27:22.172808 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.173273 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.174322 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.174481 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.174527 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:22.174688 (+ 161us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 02:27:22.192169 (+ 17481us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45927/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 02:27:22.192265 (+ 96us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43574: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45927/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 02:27:22.192365 17492 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43574: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45927/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 02:27:22.192423 17519 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:22.172399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43099 (local address 127.0.0.1:43574)
0925 02:27:22.172534 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:22.172544 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:22.172599 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:22.172716 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:22.172716 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:22.172718 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 02:27:22.172762 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:22.172764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.173333 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.173334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:22.174244 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:22.174246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.174544 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.174545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:22.174594 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:22.174599 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 02:27:22.192258 (+ 17659us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45927/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 02:27:22.192337 (+ 79us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.220.254:43099: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45927/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":94,"threads_started":1}
I20260925 02:27:22.192737 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 17421
2026-09-25T02:27:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (216 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 02:27:22.231741 12147 mini_oidc.cc:152] Starting JWKS server
I20260925 02:27:22.232270 12147 webserver.cc:460] Webserver started at https://127.0.0.1:39235/ using document root <none> and password file <none>
I20260925 02:27:22.232316 12147 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 02:27:22.232453 12147 webserver.cc:460] Webserver started at http://127.0.0.1:41775/ using document root <none> and password file <none>
2026-09-25T02:27:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:22Z Disabled control of system clock
I20260925 02:27:22.238943 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:38261
--webserver_interface=127.11.220.254
--webserver_port=0
--builtin_ntp_servers=127.11.220.212:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:38261
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39235/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 02:27:22.311081 17526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:22.311249 17526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:22.311268 17526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:22.312685 17526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:22.312719 17526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:22.312731 17526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:22.312741 17526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:22.312752 17526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 02:27:22.312762 17526 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 02:27:22.312772 17526 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39235/jwks.json
I20260925 02:27:22.313912 17526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:38261
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:38261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39235/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:22.314071 17526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:22.314271 17526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:22.318331 17526 server_base.cc:1034] running on GCE node
W20260925 02:27:22.318301 17531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:22.318295 17534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:22.318295 17532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:22.318742 17526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:22.318994 17526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:22.320139 17526 hybrid_clock.cc:648] HybridClock initialized: now 1790303242320135 us; error 29 us; skew 500 ppm
I20260925 02:27:22.321261 17526 webserver.cc:460] Webserver started at http://127.11.220.254:46815/ using document root <none> and password file <none>
I20260925 02:27:22.321460 17526 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:22.321516 17526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:22.321631 17526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:22.322599 17526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "fe0b5f6294de447d929b32e1818586ee"
format_stamp: "Formatted at 2026-09-25 02:27:22 on dist-test-slave-dn4n"
server_key: "cc011f58075c9cdcb38b98fdf0ca267a"
server_key_iv: "3ff059279e8f32e9028984f12004079a"
server_key_version: "encryptionkey@0"
I20260925 02:27:22.322943 17526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "fe0b5f6294de447d929b32e1818586ee"
format_stamp: "Formatted at 2026-09-25 02:27:22 on dist-test-slave-dn4n"
server_key: "cc011f58075c9cdcb38b98fdf0ca267a"
server_key_iv: "3ff059279e8f32e9028984f12004079a"
server_key_version: "encryptionkey@0"
I20260925 02:27:22.324102 17526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:22.324847 17540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:22.325001 17526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:22.325088 17526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "fe0b5f6294de447d929b32e1818586ee"
format_stamp: "Formatted at 2026-09-25 02:27:22 on dist-test-slave-dn4n"
server_key: "cc011f58075c9cdcb38b98fdf0ca267a"
server_key_iv: "3ff059279e8f32e9028984f12004079a"
server_key_version: "encryptionkey@0"
I20260925 02:27:22.325169 17526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:22.344391 17526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:22.344683 17526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:22.344817 17526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:22.348330 17526 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:38261
I20260925 02:27:22.348376 17593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:38261 every 8 connection(s)
I20260925 02:27:22.348680 17526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:22.349272 17594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:22.351643 17594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee: Bootstrap starting.
I20260925 02:27:22.352135 17594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:22.352406 17594 log.cc:824] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:22.352630 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 17526
I20260925 02:27:22.352733 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:22.352914 12147 external_mini_cluster.cc:1428] Setting key e62b35722d76b6f699a1b2d7dae00c50
I20260925 02:27:22.353114 17594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee: No bootstrap required, opened a new log
I20260925 02:27:22.354713 17594 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0b5f6294de447d929b32e1818586ee" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38261 } }
I20260925 02:27:22.354851 17594 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:22.354892 17594 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0b5f6294de447d929b32e1818586ee, State: Initialized, Role: FOLLOWER
I20260925 02:27:22.355005 17594 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0b5f6294de447d929b32e1818586ee" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38261 } }
I20260925 02:27:22.355095 17594 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:22.355136 17594 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:22.355180 17594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:22.355641 17594 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0b5f6294de447d929b32e1818586ee" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38261 } }
I20260925 02:27:22.355749 17594 leader_election.cc:302] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe0b5f6294de447d929b32e1818586ee; no voters:
I20260925 02:27:22.355926 17594 leader_election.cc:288] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:22.355947 17599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:22.356046 17599 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [term 1 LEADER]: Becoming Leader. State: Replica: fe0b5f6294de447d929b32e1818586ee, State: Running, Role: LEADER
I20260925 02:27:22.356129 17599 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0b5f6294de447d929b32e1818586ee" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38261 } }
I20260925 02:27:22.356169 17594 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:22.356423 17601 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe0b5f6294de447d929b32e1818586ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0b5f6294de447d929b32e1818586ee" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38261 } } }
I20260925 02:27:22.356645 17601 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:22.356577 17600 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe0b5f6294de447d929b32e1818586ee. Latest consensus state: current_term: 1 leader_uuid: "fe0b5f6294de447d929b32e1818586ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0b5f6294de447d929b32e1818586ee" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 38261 } } }
I20260925 02:27:22.356881 17600 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:22.357023 17607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:22.357450 17607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:22.358889 17607 catalog_manager.cc:1347] Generated new cluster ID: 5eb3a2e617de470f89bc7b92361c4034
I20260925 02:27:22.358944 17607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:22.359542 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:22.353390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56466 (local address 127.11.220.254:38261)
0925 02:27:22.353620 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:22.353626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:22.353648 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:27:22.354705 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:22.354711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:22.354769 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:22.354896 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:22.355095 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.356563 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.357599 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.358131 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.358243 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:22.358833 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:22.358837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:22.358838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:22.359023 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:22.359063 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:22.359076 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:22.359139 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:22.359218 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 02:27:22.359357 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":73,"threads_started":1}
I20260925 02:27:22.360363 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:27:22.368517 17607 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:22.368973 17607 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:22.374392 17607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe0b5f6294de447d929b32e1818586ee: Generated new TSK 0
I20260925 02:27:22.374531 17607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:27:22.382591 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:22.379397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56476 (local address 127.11.220.254:38261)
0925 02:27:22.379455 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:22.379456 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:22.379552 (+ 96us) server_negotiation.cc:407] Connection header received
0925 02:27:22.379594 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:22.379595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:22.379603 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:22.379650 (+ 47us) server_negotiation.cc:226] Negotiated authn=JWT
0925 02:27:22.379769 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.380184 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.381208 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.381389 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.381437 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:22.381610 (+ 173us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 02:27:22.382427 (+ 817us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39235/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 02:27:22.382538 (+ 111us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56476: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39235/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260925 02:27:22.382638 17598 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56476: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39235/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 02:27:22.382900 17623 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:22.379346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:38261 (local address 127.0.0.1:56476)
0925 02:27:22.379483 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:22.379492 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:22.379547 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:22.379668 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:22.379668 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:22.379670 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 02:27:22.379713 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:22.379715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.380267 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.380268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:22.381131 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:22.381133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.381464 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.381465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:22.381521 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:22.381526 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 02:27:22.382517 (+ 991us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39235/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 02:27:22.382623 (+ 106us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.220.254:38261: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39235/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
I20260925 02:27:22.383086 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 17526
E20260925 02:27:22.392567 17624 webserver.cc:533] Webserver: sslize failed:
E20260925 02:27:22.392634 17624 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-25T02:27:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (189 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T02:27:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:22Z Disabled control of system clock
I20260925 02:27:22.427194 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:34799
--webserver_interface=127.11.220.254
--webserver_port=0
--builtin_ntp_servers=127.11.220.212:46169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:34799
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/testchainca.cert with env {}
W20260925 02:27:22.495946 17628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:22.496097 17628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:22.496115 17628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:22.497133 17628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:22.497169 17628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:22.497179 17628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:22.497189 17628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:22.497200 17628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 02:27:22.497210 17628 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 02:27:22.497219 17628 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 02:27:22.498383 17628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:34799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:34799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:22.498550 17628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:22.498685 17628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:22.502857 17634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:22.502848 17633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:22.502835 17628 server_base.cc:1034] running on GCE node
W20260925 02:27:22.502892 17636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:22.503257 17628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:22.503536 17628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:22.504689 17628 hybrid_clock.cc:648] HybridClock initialized: now 1790303242504671 us; error 29 us; skew 500 ppm
I20260925 02:27:22.505723 17628 webserver.cc:460] Webserver started at http://127.11.220.254:46345/ using document root <none> and password file <none>
I20260925 02:27:22.505944 17628 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:22.506004 17628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:22.506136 17628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:22.507002 17628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "704781c476a24e3f991ce0978c8f9883"
format_stamp: "Formatted at 2026-09-25 02:27:22 on dist-test-slave-dn4n"
server_key: "eecf7a88b4f0162fa893ff7c379512ad"
server_key_iv: "9720602cb3e4ac29aa9448c55512c102"
server_key_version: "encryptionkey@0"
I20260925 02:27:22.507324 17628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "704781c476a24e3f991ce0978c8f9883"
format_stamp: "Formatted at 2026-09-25 02:27:22 on dist-test-slave-dn4n"
server_key: "eecf7a88b4f0162fa893ff7c379512ad"
server_key_iv: "9720602cb3e4ac29aa9448c55512c102"
server_key_version: "encryptionkey@0"
I20260925 02:27:22.508487 17628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 02:27:22.509133 17642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:22.509292 17628 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:22.509369 17628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "704781c476a24e3f991ce0978c8f9883"
format_stamp: "Formatted at 2026-09-25 02:27:22 on dist-test-slave-dn4n"
server_key: "eecf7a88b4f0162fa893ff7c379512ad"
server_key_iv: "9720602cb3e4ac29aa9448c55512c102"
server_key_version: "encryptionkey@0"
I20260925 02:27:22.509447 17628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:22.520085 17628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:22.520360 17628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:22.520543 17628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:22.524019 17628 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:34799
I20260925 02:27:22.524046 17694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:34799 every 8 connection(s)
I20260925 02:27:22.524320 17628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:22.525134 17695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:22.526895 17695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883: Bootstrap starting.
I20260925 02:27:22.527385 17695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:22.527634 17695 log.cc:824] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:22.528180 17695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883: No bootstrap required, opened a new log
I20260925 02:27:22.529611 17695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704781c476a24e3f991ce0978c8f9883" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34799 } }
I20260925 02:27:22.529726 17695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:22.529745 17695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 704781c476a24e3f991ce0978c8f9883, State: Initialized, Role: FOLLOWER
I20260925 02:27:22.529942 17695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704781c476a24e3f991ce0978c8f9883" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34799 } }
I20260925 02:27:22.530014 17695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:22.530061 17695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:22.530117 17695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:22.530506 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 17628
I20260925 02:27:22.530610 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:22.530701 17695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704781c476a24e3f991ce0978c8f9883" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34799 } }
I20260925 02:27:22.530797 12147 external_mini_cluster.cc:1428] Setting key c4e550a29eda3c0582b9d5561dbf3887
I20260925 02:27:22.530807 17695 leader_election.cc:302] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 704781c476a24e3f991ce0978c8f9883; no voters:
I20260925 02:27:22.530941 17695 leader_election.cc:288] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:22.530987 17698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:22.531169 17698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [term 1 LEADER]: Becoming Leader. State: Replica: 704781c476a24e3f991ce0978c8f9883, State: Running, Role: LEADER
I20260925 02:27:22.531251 17698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704781c476a24e3f991ce0978c8f9883" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34799 } }
I20260925 02:27:22.531308 17695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:22.531577 17698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 704781c476a24e3f991ce0978c8f9883. Latest consensus state: current_term: 1 leader_uuid: "704781c476a24e3f991ce0978c8f9883" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704781c476a24e3f991ce0978c8f9883" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34799 } } }
I20260925 02:27:22.531661 17698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:22.531589 17699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "704781c476a24e3f991ce0978c8f9883" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704781c476a24e3f991ce0978c8f9883" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34799 } } }
I20260925 02:27:22.531764 17699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:22.531909 17706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:22.532552 17706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:22.534744 17706 catalog_manager.cc:1347] Generated new cluster ID: cb14fca6a1d24a89aef23ed3136c8b43
I20260925 02:27:22.534802 17706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:22.537106 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:22.531168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40360 (local address 127.11.220.254:34799)
0925 02:27:22.531960 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:22.531968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:22.532167 (+ 199us) server_negotiation.cc:407] Connection header received
0925 02:27:22.532322 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:22.532330 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:22.532400 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:22.532563 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:22.532890 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.533777 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.534853 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.535405 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.535512 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:22.536270 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:22.536274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:22.536274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:22.536497 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:22.536532 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:22.536546 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:22.536600 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:22.536861 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 02:27:22.536927 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":55,"threads_started":1}
I20260925 02:27:22.537902 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:27:22.545987 17706 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:22.546586 17706 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:22.550437 17706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 704781c476a24e3f991ce0978c8f9883: Generated new TSK 0
I20260925 02:27:22.550575 17706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:27:22.592959 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:22.556814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40376 (local address 127.11.220.254:34799)
0925 02:27:22.556855 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:22.556856 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:22.556936 (+ 80us) server_negotiation.cc:407] Connection header received
0925 02:27:22.556995 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:22.556995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:22.557004 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:22.557055 (+ 51us) server_negotiation.cc:226] Negotiated authn=JWT
0925 02:27:22.557186 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.557619 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.558611 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.558764 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.558806 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:22.558968 (+ 162us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 02:27:22.592796 (+ 33828us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 02:27:22.592910 (+ 114us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40376: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 02:27:22.593027 17702 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40376: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 02:27:22.593123 17725 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:22.556712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:34799 (local address 127.0.0.1:40376)
0925 02:27:22.556871 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:22.556882 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:22.556941 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:22.557068 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:22.557069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:22.557071 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0925 02:27:22.557119 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:22.557121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.557688 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.557688 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:22.558536 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:22.558538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:22.558826 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:22.558827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:22.558879 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:22.558884 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 02:27:22.592908 (+ 34024us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 02:27:22.593023 (+ 115us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.220.254:34799: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260925 02:27:22.593443 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 17628
2026-09-25T02:27:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (189 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (80 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (79 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:22 dist-test-slave-dn4n krb5kdc[17737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:22 dist-test-slave-dn4n krb5kdc[17737](info): set up 2 sockets
Sep 25 02:27:22 dist-test-slave-dn4n krb5kdc[17737](info): commencing operation
krb5kdc: starting...
W20260925 02:27:23.684746 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.913s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:23 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:23.784945 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:34047
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:43695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:34047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:23.856472 17750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:23.856678 17750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:23.856709 17750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:23.857684 17750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:23.857736 17750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:23.857755 17750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:23.857772 17750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:23.857798 17750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:23.859069 17750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:34047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:34047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:23.859339 17750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:23.859562 17750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:23.863828 17755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:23.863860 17756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:23.863870 17758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:23.863953 17750 server_base.cc:1034] running on GCE node
I20260925 02:27:23.864213 17750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:23.864523 17750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:23.865672 17750 hybrid_clock.cc:648] HybridClock initialized: now 1790303243865659 us; error 23 us; skew 500 ppm
Sep 25 02:27:23 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303243, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:23.867731 17750 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:23.868152 17750 webserver.cc:460] Webserver started at http://127.11.220.254:36247/ using document root <none> and password file <none>
I20260925 02:27:23.868363 17750 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:23.868427 17750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:23.868587 17750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:23.869539 17750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "7c98530aabf84b46983384545d233550"
format_stamp: "Formatted at 2026-09-25 02:27:23 on dist-test-slave-dn4n"
server_key: "09369eee3d84a90db4e57a9761206820"
server_key_iv: "4ea381efd8a2fe6a808d511f740b4463"
server_key_version: "encryptionkey@0"
I20260925 02:27:23.869860 17750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "7c98530aabf84b46983384545d233550"
format_stamp: "Formatted at 2026-09-25 02:27:23 on dist-test-slave-dn4n"
server_key: "09369eee3d84a90db4e57a9761206820"
server_key_iv: "4ea381efd8a2fe6a808d511f740b4463"
server_key_version: "encryptionkey@0"
I20260925 02:27:23.871045 17750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:23.871691 17765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:23.871852 17750 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:23.871940 17750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "7c98530aabf84b46983384545d233550"
format_stamp: "Formatted at 2026-09-25 02:27:23 on dist-test-slave-dn4n"
server_key: "09369eee3d84a90db4e57a9761206820"
server_key_iv: "4ea381efd8a2fe6a808d511f740b4463"
server_key_version: "encryptionkey@0"
I20260925 02:27:23.872018 17750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:23.879974 17750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:23.882004 17750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:23.882156 17750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:23.885924 17750 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:34047
I20260925 02:27:23.885962 17817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:34047 every 8 connection(s)
I20260925 02:27:23.886273 17750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:23.887028 17818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:23.888620 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 17750
I20260925 02:27:23.888731 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:23.888917 12147 external_mini_cluster.cc:1428] Setting key 231cb4c417ae83279ecf50bd4b0a420a
I20260925 02:27:23.889307 17818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550: Bootstrap starting.
I20260925 02:27:23.889899 17818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:23.890221 17818 log.cc:824] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:23.890929 17818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550: No bootstrap required, opened a new log
I20260925 02:27:23.892997 17818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c98530aabf84b46983384545d233550" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34047 } }
I20260925 02:27:23.893150 17818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:23.893174 17818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c98530aabf84b46983384545d233550, State: Initialized, Role: FOLLOWER
I20260925 02:27:23.893327 17818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c98530aabf84b46983384545d233550" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34047 } }
I20260925 02:27:23.893394 17818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:23.893415 17818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:23.893478 17818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:23.893997 17818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c98530aabf84b46983384545d233550" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34047 } }
I20260925 02:27:23.894076 17818 leader_election.cc:302] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c98530aabf84b46983384545d233550; no voters:
I20260925 02:27:23.894251 17818 leader_election.cc:288] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:23.894320 17823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:23.894500 17823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [term 1 LEADER]: Becoming Leader. State: Replica: 7c98530aabf84b46983384545d233550, State: Running, Role: LEADER
I20260925 02:27:23.894500 17818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:23.894634 17823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c98530aabf84b46983384545d233550" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34047 } }
I20260925 02:27:23.894919 17823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c98530aabf84b46983384545d233550. Latest consensus state: current_term: 1 leader_uuid: "7c98530aabf84b46983384545d233550" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c98530aabf84b46983384545d233550" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34047 } } }
I20260925 02:27:23.895002 17823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:23.894930 17824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c98530aabf84b46983384545d233550" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c98530aabf84b46983384545d233550" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34047 } } }
I20260925 02:27:23.895123 17824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550 [sys.catalog]: This master's current role is: LEADER
Sep 25 02:27:23 dist-test-slave-dn4n krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
W20260925 02:27:23.896325 17839 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:27:23.896386 17839 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:27:23.896497 17830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:23.896687 17830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:23.898394 17830 catalog_manager.cc:1347] Generated new cluster ID: f2692c422f6540e592d105f1e5640276
I20260925 02:27:23.898447 17830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:23.900055 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:23.889399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60910 (local address 127.11.220.254:34047)
0925 02:27:23.889601 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:23.889608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:23.889631 (+ 23us) server_negotiation.cc:407] Connection header received
0925 02:27:23.890059 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:23.890065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:23.890108 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:23.890214 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:23.890991 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:23.891864 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:23.893015 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:23.893612 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:23.893740 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:23.896664 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:23.896668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:23.896668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:23.896680 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:23.898515 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:23.898855 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:23.898858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:23.898858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:23.898949 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:23.899147 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:23.899148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:23.899148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:23.899368 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:23.899562 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:23.899695 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 02:27:23.899846 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":118,"threads_started":1}
I20260925 02:27:23.906875 17830 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:23.907296 17830 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:23.912609 17830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c98530aabf84b46983384545d233550: Generated new TSK 0
I20260925 02:27:23.912770 17830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:23.940958 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34047
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:43695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:24.015141 17844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:24.015314 17844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:24.015334 17844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:24.016467 17844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:24.016505 17844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:24.016553 17844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:24.017896 17844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:24.018106 17844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:24.018316 17844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:24.022765 17849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:24.022781 17850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:24.022809 17844 server_base.cc:1034] running on GCE node
W20260925 02:27:24.022781 17852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:24.023218 17844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:24.023459 17844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:24.024602 17844 hybrid_clock.cc:648] HybridClock initialized: now 1790303244024587 us; error 26 us; skew 500 ppm
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:24.026870 17844 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:24.027274 17844 webserver.cc:460] Webserver started at http://127.11.220.193:43357/ using document root <none> and password file <none>
I20260925 02:27:24.027513 17844 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:24.027576 17844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:24.027709 17844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:24.029320 17844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "aa714b6c1ca34350bd983495aac7d96c"
format_stamp: "Formatted at 2026-09-25 02:27:24 on dist-test-slave-dn4n"
server_key: "07546998ff9dc237c486397865940d10"
server_key_iv: "bf3a44781ee7bbe7cf845d3140d621c0"
server_key_version: "encryptionkey@0"
I20260925 02:27:24.029727 17844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "aa714b6c1ca34350bd983495aac7d96c"
format_stamp: "Formatted at 2026-09-25 02:27:24 on dist-test-slave-dn4n"
server_key: "07546998ff9dc237c486397865940d10"
server_key_iv: "bf3a44781ee7bbe7cf845d3140d621c0"
server_key_version: "encryptionkey@0"
I20260925 02:27:24.031687 17844 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 02:27:24.032681 17859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:24.033128 17844 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:24.033237 17844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "aa714b6c1ca34350bd983495aac7d96c"
format_stamp: "Formatted at 2026-09-25 02:27:24 on dist-test-slave-dn4n"
server_key: "07546998ff9dc237c486397865940d10"
server_key_iv: "bf3a44781ee7bbe7cf845d3140d621c0"
server_key_version: "encryptionkey@0"
I20260925 02:27:24.033316 17844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:24.054777 17844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:24.056769 17844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:24.056861 17844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:24.057003 17844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:24.057291 17844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:24.057322 17844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:24.057370 17844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:24.057394 17844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:24.062460 17844 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:37579
I20260925 02:27:24.062526 17973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:37579 every 8 connection(s)
I20260925 02:27:24.062812 17844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:24.064899 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 17844
I20260925 02:27:24.065001 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:24.065176 12147 external_mini_cluster.cc:1428] Setting key 2d7e43b2d5b7e81deeac13524fbe273a
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:24.071823 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:24.063440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:60277 (local address 127.11.220.254:34047)
0925 02:27:24.063499 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:24.063500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:24.063683 (+ 183us) server_negotiation.cc:407] Connection header received
0925 02:27:24.064586 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:24.064587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:24.064597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:24.064650 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:24.065424 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.065894 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.066788 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.066981 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.067061 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:24.069325 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:24.069327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:24.069328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:24.069335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:24.070567 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.070944 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.070945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.070946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.071003 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.071240 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.071241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.071242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.071348 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:24.071410 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:24.071655 (+ 245us) server_negotiation.cc:299] Negotiation successful
0925 02:27:24.071735 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 02:27:24.071985 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:24.063427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:34047 (local address 127.11.220.193:60277)
0925 02:27:24.063566 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:24.063602 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:24.064442 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:24.064700 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:24.064703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:24.065064 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:24.065322 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:24.065329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.065983 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.065984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:24.066702 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:24.066704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.067073 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.067074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:24.067190 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:24.067636 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:24.067646 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:24.069225 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:24.070645 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:24.070652 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:24.070654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:24.070874 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:24.071065 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:24.071066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:24.071067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:24.071172 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:24.071413 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:24.071420 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:24.071540 (+ 120us) client_negotiation.cc:769] Sending connection context
0925 02:27:24.071635 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 02:27:24.071780 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":78,"threads_started":1}
I20260925 02:27:24.072387 17974 heartbeater.cc:347] Connected to a master server at 127.11.220.254:34047
I20260925 02:27:24.072494 17974 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:24.072667 17974 heartbeater.cc:511] Master 127.11.220.254:34047 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
I20260925 02:27:24.073144 17782 ts_manager.cc:194] Registered new tserver with Master: aa714b6c1ca34350bd983495aac7d96c (127.11.220.193:37579)
I20260925 02:27:24.073717 17782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:60277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:24.089015 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34047
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:43695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:24.158993 17979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:24.159163 17979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:24.159183 17979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:24.160095 17979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:24.160128 17979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:24.160151 17979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:24.161358 17979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:24.161504 17979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:24.161640 17979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:24.166067 17985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:24.166085 17987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:24.166100 17984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:24.166314 17979 server_base.cc:1034] running on GCE node
I20260925 02:27:24.166491 17979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:24.166653 17979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:24.167816 17979 hybrid_clock.cc:648] HybridClock initialized: now 1790303244167794 us; error 35 us; skew 500 ppm
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:24.170042 17979 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:24.170449 17979 webserver.cc:460] Webserver started at http://127.11.220.194:39645/ using document root <none> and password file <none>
I20260925 02:27:24.170641 17979 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:24.170701 17979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:24.170832 17979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:24.171738 17979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "4be20acc46354b76b78a1e8f4601c74a"
format_stamp: "Formatted at 2026-09-25 02:27:24 on dist-test-slave-dn4n"
server_key: "bc0ad1141c6beae4e3cd500d2bb6cbd1"
server_key_iv: "314f73596dcfd08c90e5897d255549d6"
server_key_version: "encryptionkey@0"
I20260925 02:27:24.172046 17979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "4be20acc46354b76b78a1e8f4601c74a"
format_stamp: "Formatted at 2026-09-25 02:27:24 on dist-test-slave-dn4n"
server_key: "bc0ad1141c6beae4e3cd500d2bb6cbd1"
server_key_iv: "314f73596dcfd08c90e5897d255549d6"
server_key_version: "encryptionkey@0"
I20260925 02:27:24.173374 17979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:24.174085 17994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:24.174249 17979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:24.174347 17979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "4be20acc46354b76b78a1e8f4601c74a"
format_stamp: "Formatted at 2026-09-25 02:27:24 on dist-test-slave-dn4n"
server_key: "bc0ad1141c6beae4e3cd500d2bb6cbd1"
server_key_iv: "314f73596dcfd08c90e5897d255549d6"
server_key_version: "encryptionkey@0"
I20260925 02:27:24.174429 17979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:24.183610 17979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:24.185698 17979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:24.185864 17979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:24.186074 17979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:24.186339 17979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:24.186388 17979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:24.186424 17979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:24.186450 17979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:24.191679 17979 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:40277
I20260925 02:27:24.191711 18109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:40277 every 8 connection(s)
I20260925 02:27:24.191977 17979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:24.192854 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 17979
I20260925 02:27:24.192962 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:24.193125 12147 external_mini_cluster.cc:1428] Setting key 9620fb3e3641c0cec9e77a27019ce1fb
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
I20260925 02:27:24.201534 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:24.192627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:45235 (local address 127.11.220.254:34047)
0925 02:27:24.192663 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:24.192665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:24.192837 (+ 172us) server_negotiation.cc:407] Connection header received
0925 02:27:24.193866 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:24.193868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:24.193880 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:24.193919 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:24.194475 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.195232 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.196282 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.196625 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.196710 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:24.198749 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:24.198750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:24.198750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:24.198759 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:24.200211 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.200639 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.200641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.200642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.200707 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.200925 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.200926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.200926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.201057 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:24.201130 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:24.201338 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 02:27:24.201425 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:27:24.201630 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:24.192600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:34047 (local address 127.11.220.194:45235)
0925 02:27:24.192725 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:24.192768 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:24.193782 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:24.193976 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:24.193979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:24.194208 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:24.194372 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:24.194376 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.195348 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.195350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:24.196175 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:24.196178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.196715 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.196716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:24.196821 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:24.197296 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:24.197306 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:24.198665 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:24.200286 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:24.200292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:24.200294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:24.200572 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:24.200775 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:24.200776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:24.200776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:24.200862 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:24.201124 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:24.201128 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:24.201221 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 02:27:24.201292 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 02:27:24.201416 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260925 02:27:24.202021 18110 heartbeater.cc:347] Connected to a master server at 127.11.220.254:34047
I20260925 02:27:24.202092 18110 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:24.202200 18110 heartbeater.cc:511] Master 127.11.220.254:34047 requested a full tablet report, sending...
I20260925 02:27:24.202517 17782 ts_manager.cc:194] Registered new tserver with Master: 4be20acc46354b76b78a1e8f4601c74a (127.11.220.194:40277)
I20260925 02:27:24.202898 17782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:45235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:24.217504 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34047
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:43695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:24.291815 18115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:24.292016 18115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:24.292053 18115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:24.293183 18115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:24.293239 18115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:24.293282 18115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:24.294673 18115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:34047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:24.294914 18115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:24.295111 18115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:24.299345 18121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:24.299350 18120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:24.299360 18123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:24.299579 18115 server_base.cc:1034] running on GCE node
I20260925 02:27:24.299819 18115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:24.300050 18115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:24.301204 18115 hybrid_clock.cc:648] HybridClock initialized: now 1790303244301194 us; error 27 us; skew 500 ppm
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:24.303315 18115 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:24.303761 18115 webserver.cc:460] Webserver started at http://127.11.220.195:33481/ using document root <none> and password file <none>
I20260925 02:27:24.303975 18115 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:24.304040 18115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:24.304186 18115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:24.305140 18115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "0395ab12dcdf47239e3208e1bf0dc373"
format_stamp: "Formatted at 2026-09-25 02:27:24 on dist-test-slave-dn4n"
server_key: "51e02409c03dd86775a58c3e0ad5ccf9"
server_key_iv: "7ff2c82e412fd2dd4d121829823217f6"
server_key_version: "encryptionkey@0"
I20260925 02:27:24.305465 18115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "0395ab12dcdf47239e3208e1bf0dc373"
format_stamp: "Formatted at 2026-09-25 02:27:24 on dist-test-slave-dn4n"
server_key: "51e02409c03dd86775a58c3e0ad5ccf9"
server_key_iv: "7ff2c82e412fd2dd4d121829823217f6"
server_key_version: "encryptionkey@0"
I20260925 02:27:24.306665 18115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:27:24.307278 18130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:24.307443 18115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:24.307545 18115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "0395ab12dcdf47239e3208e1bf0dc373"
format_stamp: "Formatted at 2026-09-25 02:27:24 on dist-test-slave-dn4n"
server_key: "51e02409c03dd86775a58c3e0ad5ccf9"
server_key_iv: "7ff2c82e412fd2dd4d121829823217f6"
server_key_version: "encryptionkey@0"
I20260925 02:27:24.307639 18115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:24.346414 18115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:24.348026 18115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:24.348181 18115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:24.348387 18115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:24.348687 18115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:24.348738 18115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:24.348775 18115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:24.348799 18115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:24.353981 18115 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:38273
I20260925 02:27:24.354024 18243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:38273 every 8 connection(s)
I20260925 02:27:24.354306 18115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:24.361563 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 18115
I20260925 02:27:24.361665 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:24.361871 12147 external_mini_cluster.cc:1428] Setting key 7bca0e23ea17f24d5f8fa61420ffe6d3
I20260925 02:27:24.362744 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:24.354964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:49127 (local address 127.11.220.254:34047)
0925 02:27:24.355010 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:24.355012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:24.355199 (+ 187us) server_negotiation.cc:407] Connection header received
0925 02:27:24.355979 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:24.355981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:24.355989 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:24.356036 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:24.356663 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.357155 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.358058 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.358241 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.358285 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:24.359992 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:24.359994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:24.359994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:24.360000 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:24.361291 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.361735 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.361737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.361737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.361797 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.362059 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.362061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.362061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.362190 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:24.362266 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:24.362568 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 02:27:24.362623 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:27:24.362911 18246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:24.354944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:34047 (local address 127.11.220.195:49127)
0925 02:27:24.355085 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:24.355128 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:24.355904 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:24.356119 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:24.356122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:24.356344 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:24.356589 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:24.356593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.357234 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.357235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:24.357992 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:24.357993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.358304 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.358304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:24.358388 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:24.358780 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:24.358787 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:24.359917 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:24.361348 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:24.361354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:24.361355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:24.361637 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:24.361867 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:24.361868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:24.361868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:24.361984 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:24.362263 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:24.362268 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:24.362433 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 02:27:24.362538 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 02:27:24.362681 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260925 02:27:24.363271 18244 heartbeater.cc:347] Connected to a master server at 127.11.220.254:34047
I20260925 02:27:24.363364 18244 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:24.363533 18244 heartbeater.cc:511] Master 127.11.220.254:34047 requested a full tablet report, sending...
I20260925 02:27:24.363857 17782 ts_manager.cc:194] Registered new tserver with Master: 0395ab12dcdf47239e3208e1bf0dc373 (127.11.220.195:38273)
I20260925 02:27:24.364205 17782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:49127
I20260925 02:27:24.373646 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/test-admin.keytab.
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:24.388295 12147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 02:27:24.389721 18251 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:24.389075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.11.220.193:37579)
0925 02:27:24.389227 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:24.389233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:24.389246 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:27:24.389267 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:24.389268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:24.389280 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:24.389342 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:24.389666 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53804: BlockingRecv error: recv got EOF from 127.0.0.1:53804 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
W20260925 02:27:24.389837 12147 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5cc' of type FILE
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:24.390707 12147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:24.391467 12147 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:27:24.398180 18251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:24.391922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53810 (local address 127.11.220.193:37579)
0925 02:27:24.391961 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:24.391963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:24.391971 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:24.392035 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:24.392035 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:24.392044 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:24.392085 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:24.392399 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.393099 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.393737 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.394067 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.394124 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:24.396042 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:24.396050 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:24.396050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:24.396058 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:24.397362 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.397644 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.397646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.397646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.397694 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.397828 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.397828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.397828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.397924 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:24.398017 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:24.398063 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 02:27:24.398122 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 02:27:24.399645 18251 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:24.399187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.11.220.193:37579)
0925 02:27:24.399234 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:24.399236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:24.399322 (+ 86us) server_negotiation.cc:407] Connection header received
0925 02:27:24.399343 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:24.399344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:24.399351 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:24.399390 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:24.399587 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53822: BlockingRecv error: recv got EOF from 127.0.0.1:53822 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260925 02:27:24.399772 12147 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5cc)
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:24.400652 12147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:24.401372 12147 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:27:24.406730 18251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:24.401753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53836 (local address 127.11.220.193:37579)
0925 02:27:24.401803 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:24.401804 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:24.401814 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:27:24.401870 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:24.401871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:24.401878 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:24.401916 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:24.402204 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.402598 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.403139 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.403301 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.403344 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:24.404975 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:24.404976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:24.404976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:24.404981 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:24.405978 (+ 997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.406253 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.406254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.406254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.406298 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.406397 (+ 99us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.406397 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.406398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.406457 (+ 59us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:24.406512 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:24.406566 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 02:27:24.406613 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260925 02:27:24.407989 18251 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:24.407607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.11.220.193:37579)
0925 02:27:24.407635 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:24.407637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:24.407692 (+ 55us) server_negotiation.cc:407] Connection header received
0925 02:27:24.407743 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:24.407744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:24.407750 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:24.407791 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:24.407962 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53838: BlockingRecv error: recv got EOF from 127.0.0.1:53838 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 02:27:24.408119 12147 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790303222243611-12147-0/krb5kdc/krb5cc' of type FILE
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:24.409024 12147 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:24.409979 12147 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[17737](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:27:24.415908 18251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:24.410409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.11.220.193:37579)
0925 02:27:24.410459 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:24.410461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:24.410550 (+ 89us) server_negotiation.cc:407] Connection header received
0925 02:27:24.410613 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:24.410614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:24.410622 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:24.410671 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:24.411068 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.411467 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.412109 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:24.412280 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:24.412353 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:24.414215 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:24.414216 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:24.414217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:24.414222 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:24.415260 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.415510 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.415510 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.415510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.415548 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:24.415656 (+ 108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:24.415657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:24.415657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:24.415716 (+ 59us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:24.415765 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:24.415820 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 02:27:24.415859 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 02:27:24.416523 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 17844
I20260925 02:27:24.422611 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 17979
I20260925 02:27:24.427069 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 18115
I20260925 02:27:24.431584 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 17750
2026-09-25T02:27:24Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1679 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[18264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[18264](info): set up 2 sockets
Sep 25 02:27:24 dist-test-slave-dn4n krb5kdc[18264](info): commencing operation
krb5kdc: starting...
W20260925 02:27:25.377031 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:25 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.254.keytab.
Entry for principal oryx/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.254.keytab.
I20260925 02:27:25.477432 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:33109
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.254.keytab
--principal=oryx/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:33109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:25.548995 18277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:25.549201 18277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:25.549232 18277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:25.550242 18277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:25.550299 18277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:25.550318 18277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:25.550336 18277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:25.550361 18277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:25.551594 18277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:33109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.254.keytab
--principal=oryx/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:33109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:25.551834 18277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:25.552027 18277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:25.556533 18282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:25.556627 18277 server_base.cc:1034] running on GCE node
W20260925 02:27:25.556533 18285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:25.556552 18283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:25.556957 18277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:25.557212 18277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:25.558342 18277 hybrid_clock.cc:648] HybridClock initialized: now 1790303245558333 us; error 26 us; skew 500 ppm
Sep 25 02:27:25 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:25.560527 18277 init.cc:377] Logged in from keytab as oryx/127.11.220.254@KRBTEST.COM (short username oryx)
I20260925 02:27:25.560995 18277 webserver.cc:460] Webserver started at http://127.11.220.254:35765/ using document root <none> and password file <none>
I20260925 02:27:25.561213 18277 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:25.561272 18277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:25.561390 18277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:25.562270 18277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "d0e02c0ed8aa4340be749d8f7b461dea"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "ffe34e7d8f1017cce10dc732dd743f6e"
server_key_iv: "1ea4b7e957028a849edbb1930fbc14db"
server_key_version: "encryptionkey@0"
I20260925 02:27:25.562587 18277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "d0e02c0ed8aa4340be749d8f7b461dea"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "ffe34e7d8f1017cce10dc732dd743f6e"
server_key_iv: "1ea4b7e957028a849edbb1930fbc14db"
server_key_version: "encryptionkey@0"
I20260925 02:27:25.563731 18277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:25.564394 18292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:25.564558 18277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:25.564671 18277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "d0e02c0ed8aa4340be749d8f7b461dea"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "ffe34e7d8f1017cce10dc732dd743f6e"
server_key_iv: "1ea4b7e957028a849edbb1930fbc14db"
server_key_version: "encryptionkey@0"
I20260925 02:27:25.564747 18277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:25.574590 18277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:25.576118 18277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:25.576256 18277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:25.579799 18277 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:33109
I20260925 02:27:25.579829 18344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:33109 every 8 connection(s)
I20260925 02:27:25.580142 18277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:25.581073 18345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:25.581067 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 18277
I20260925 02:27:25.581199 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:25.581387 12147 external_mini_cluster.cc:1428] Setting key d5c96457a53a3de6cb27ed18f75e1544
I20260925 02:27:25.584129 18345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea: Bootstrap starting.
I20260925 02:27:25.584787 18345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:25.585048 18345 log.cc:824] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:25.585654 18345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea: No bootstrap required, opened a new log
Sep 25 02:27:25 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:25.587472 18345 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } }
I20260925 02:27:25.587621 18345 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:25.587664 18345 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e02c0ed8aa4340be749d8f7b461dea, State: Initialized, Role: FOLLOWER
I20260925 02:27:25.587788 18345 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } }
I20260925 02:27:25.587880 18345 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:25.587915 18345 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:25.587966 18345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:25.588744 18345 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } }
I20260925 02:27:25.588877 18345 leader_election.cc:302] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0e02c0ed8aa4340be749d8f7b461dea; no voters:
I20260925 02:27:25.589027 18345 leader_election.cc:288] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:25.589063 18350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:25.589169 18350 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 1 LEADER]: Becoming Leader. State: Replica: d0e02c0ed8aa4340be749d8f7b461dea, State: Running, Role: LEADER
I20260925 02:27:25.589248 18350 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } }
I20260925 02:27:25.589278 18345 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:25.589587 18350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0e02c0ed8aa4340be749d8f7b461dea. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } }
I20260925 02:27:25.589690 18350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:25.589608 18352 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } }
I20260925 02:27:25.589859 18352 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:25.589915 18356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:25.590394 18356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:25.591794 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:25.581848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37926 (local address 127.11.220.254:33109)
0925 02:27:25.582018 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:25.582026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:25.582054 (+ 28us) server_negotiation.cc:407] Connection header received
0925 02:27:25.582488 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:25.582496 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:25.582551 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:25.582712 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:25.583201 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:25.584538 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:25.585265 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:25.585642 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:25.585765 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:25.587832 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:25.587836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:25.587836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:25.587847 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:25.589909 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:25.590257 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:25.590260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:25.590260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:25.590352 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:25.590613 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:25.590614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:25.590614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:25.590884 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:25.591033 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:25.591436 (+ 403us) server_negotiation.cc:299] Negotiation successful
0925 02:27:25.591567 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260925 02:27:25.591830 18367 mvcc.cc:205] Tried to move back new op lower bound from 7333082093941895168 to 7333082093933924352. Current Snapshot: MvccSnapshot[applied={T|T < 7333082093941895168}]
I20260925 02:27:25.592474 18356 catalog_manager.cc:1347] Generated new cluster ID: 013950402d7b4ece8b5dc4d4425796f1
I20260925 02:27:25.592525 18356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:25.614812 18356 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:25.615440 18356 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:25.620707 18356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea: Generated new TSK 0
I20260925 02:27:25.620875 18356 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab.
Entry for principal oryx/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab.
I20260925 02:27:25.651019 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33109
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab
--principal=oryx/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:25.739296 18371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:25.739461 18371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:25.739481 18371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:25.740533 18371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:25.740567 18371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:25.740615 18371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:25.741880 18371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab
--principal=oryx/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:25.742097 18371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:25.742316 18371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:25.746719 18377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:25.746727 18379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:25.746738 18376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:25.746857 18371 server_base.cc:1034] running on GCE node
I20260925 02:27:25.747022 18371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:25.747189 18371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:25.748315 18371 hybrid_clock.cc:648] HybridClock initialized: now 1790303245748292 us; error 37 us; skew 500 ppm
Sep 25 02:27:25 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:25.750470 18371 init.cc:377] Logged in from keytab as oryx/127.11.220.193@KRBTEST.COM (short username oryx)
I20260925 02:27:25.750792 18371 webserver.cc:460] Webserver started at http://127.11.220.193:41025/ using document root <none> and password file <none>
I20260925 02:27:25.750945 18371 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:25.750979 18371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:25.751057 18371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:25.751878 18371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "c4236f9f8b9f42de9809d168a1ee5bf6"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "acf0ac71024e80a0a75ec25165a41421"
server_key_iv: "8fa0f70381c08e70dc74587e406968a1"
server_key_version: "encryptionkey@0"
I20260925 02:27:25.752137 18371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "c4236f9f8b9f42de9809d168a1ee5bf6"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "acf0ac71024e80a0a75ec25165a41421"
server_key_iv: "8fa0f70381c08e70dc74587e406968a1"
server_key_version: "encryptionkey@0"
I20260925 02:27:25.753490 18371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:27:25.754081 18386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:25.754241 18371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:25.754304 18371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "c4236f9f8b9f42de9809d168a1ee5bf6"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "acf0ac71024e80a0a75ec25165a41421"
server_key_iv: "8fa0f70381c08e70dc74587e406968a1"
server_key_version: "encryptionkey@0"
I20260925 02:27:25.754352 18371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:25.762548 18371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:25.764110 18371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:25.764209 18371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:25.764359 18371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:25.764694 18371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:25.764724 18371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:25.764742 18371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:25.764753 18371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:25.769747 18371 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:43443
I20260925 02:27:25.769814 18500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:43443 every 8 connection(s)
I20260925 02:27:25.770059 18371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
Sep 25 02:27:25 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:25.775538 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 18371
I20260925 02:27:25.775626 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:25.775792 12147 external_mini_cluster.cc:1428] Setting key 86da865b2864aa8a8d74e87b4f8e3e0b
I20260925 02:27:25.778553 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:25.770633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:42597 (local address 127.11.220.254:33109)
0925 02:27:25.770670 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:25.770672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:25.770878 (+ 206us) server_negotiation.cc:407] Connection header received
0925 02:27:25.771308 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:25.771310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:25.771322 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:25.771363 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:25.772228 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:25.772787 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:25.773721 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:25.773923 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:25.773993 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:25.775886 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:25.775888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:25.775888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:25.775895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:25.777338 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:25.777674 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:25.777675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:25.777676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:25.777732 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:25.778006 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:25.778006 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:25.778006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:25.778097 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:25.778157 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:25.778440 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 02:27:25.778496 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:27:25.778885 18503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:25.770626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.193:42597)
0925 02:27:25.770778 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:25.770807 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:25.771212 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:25.771428 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:25.771432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:25.771971 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:25.772135 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:25.772139 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:25.772871 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:25.772872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:25.773627 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:25.773629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:25.774001 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:25.774002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:25.774126 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:25.774546 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:25.774565 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:25.775784 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:25.777415 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:25.777421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:25.777422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:25.777607 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:25.777803 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:25.777804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:25.777804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:25.777943 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:25.778169 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:25.778174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:25.778317 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 02:27:25.778421 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 02:27:25.778589 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260925 02:27:25.779307 18501 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33109
I20260925 02:27:25.779392 18501 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:25.779544 18501 heartbeater.cc:511] Master 127.11.220.254:33109 requested a full tablet report, sending...
I20260925 02:27:25.779973 18309 ts_manager.cc:194] Registered new tserver with Master: c4236f9f8b9f42de9809d168a1ee5bf6 (127.11.220.193:43443)
I20260925 02:27:25.780706 18309 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:42597
WARNING: no policy specified for oryx/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab.
Entry for principal oryx/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab.
I20260925 02:27:25.798780 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33109
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab
--principal=oryx/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:25.870891 18506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:25.871054 18506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:25.871075 18506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:25.872108 18506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:25.872143 18506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:25.872174 18506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:25.873533 18506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab
--principal=oryx/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:25.873735 18506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:25.873967 18506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:25.878391 18506 server_base.cc:1034] running on GCE node
W20260925 02:27:25.878337 18513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:25.878383 18512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:25.878337 18515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:25.878682 18506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:25.878901 18506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:25.880059 18506 hybrid_clock.cc:648] HybridClock initialized: now 1790303245880045 us; error 29 us; skew 500 ppm
Sep 25 02:27:25 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:25.882225 18506 init.cc:377] Logged in from keytab as oryx/127.11.220.194@KRBTEST.COM (short username oryx)
I20260925 02:27:25.882570 18506 webserver.cc:460] Webserver started at http://127.11.220.194:34879/ using document root <none> and password file <none>
I20260925 02:27:25.882762 18506 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:25.882817 18506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:25.882922 18506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:25.883860 18506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "210baf669be6468a818f21f2e0f17139"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "e5cb8602f9bd3435100edfbd92dd82af"
server_key_iv: "82d36b921ab4d726e25e2a102ea3d1eb"
server_key_version: "encryptionkey@0"
I20260925 02:27:25.884202 18506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "210baf669be6468a818f21f2e0f17139"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "e5cb8602f9bd3435100edfbd92dd82af"
server_key_iv: "82d36b921ab4d726e25e2a102ea3d1eb"
server_key_version: "encryptionkey@0"
I20260925 02:27:25.885519 18506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:27:25.886211 18522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:25.886365 18506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:25.886458 18506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "210baf669be6468a818f21f2e0f17139"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "e5cb8602f9bd3435100edfbd92dd82af"
server_key_iv: "82d36b921ab4d726e25e2a102ea3d1eb"
server_key_version: "encryptionkey@0"
I20260925 02:27:25.886528 18506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:25.900776 18506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:25.902300 18506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:25.902455 18506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:25.902635 18506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:25.902921 18506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:25.902967 18506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:25.903002 18506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:25.903024 18506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:25.908066 18506 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:46147
I20260925 02:27:25.908176 18636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:46147 every 8 connection(s)
I20260925 02:27:25.908540 18506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:25.912569 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 18506
I20260925 02:27:25.912680 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:25.912858 12147 external_mini_cluster.cc:1428] Setting key cfe1ac28d3971e1f3a24f597b8f7a885
Sep 25 02:27:25 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:25.917420 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:25.909515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:56857 (local address 127.11.220.254:33109)
0925 02:27:25.909557 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:25.909560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:25.909763 (+ 203us) server_negotiation.cc:407] Connection header received
0925 02:27:25.910157 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:25.910159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:25.910169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:25.910217 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:25.911018 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:25.911491 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:25.912313 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:25.912582 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:25.912653 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:25.914997 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:25.914998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:25.914998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:25.915004 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:25.916058 (+ 1054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:25.916502 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:25.916503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:25.916503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:25.916556 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:25.916801 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:25.916801 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:25.916802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:25.916922 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:25.916996 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:25.917304 (+ 308us) server_negotiation.cc:299] Negotiation successful
0925 02:27:25.917365 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:25.917678 18639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:25.909499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.194:56857)
0925 02:27:25.909650 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:25.909685 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:25.910052 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:25.910262 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:25.910265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:25.910792 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:25.910951 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:25.910956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:25.911565 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:25.911566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:25.912248 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:25.912250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:25.912677 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:25.912679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:25.912806 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:25.913247 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:25.913261 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:25.914911 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:25.916127 (+ 1216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:25.916133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:25.916135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:25.916404 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:25.916614 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:25.916615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:25.916616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:25.916743 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:25.917013 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:25.917020 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:25.917179 (+ 159us) client_negotiation.cc:769] Sending connection context
0925 02:27:25.917286 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 02:27:25.917431 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260925 02:27:25.918207 18637 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33109
I20260925 02:27:25.918298 18637 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:25.918478 18637 heartbeater.cc:511] Master 127.11.220.254:33109 requested a full tablet report, sending...
I20260925 02:27:25.918813 18309 ts_manager.cc:194] Registered new tserver with Master: 210baf669be6468a818f21f2e0f17139 (127.11.220.194:46147)
I20260925 02:27:25.919148 18309 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:56857
WARNING: no policy specified for oryx/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab.
Entry for principal oryx/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab.
I20260925 02:27:25.934688 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33109
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab
--principal=oryx/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:26.009268 18642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:26.009430 18642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:26.009451 18642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:26.010463 18642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:26.010497 18642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:26.010521 18642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:26.011821 18642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab
--principal=oryx/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:26.011968 18642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:26.012172 18642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:26.016619 18648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.016661 18650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.016822 18642 server_base.cc:1034] running on GCE node
W20260925 02:27:26.016660 18647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.017088 18642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:26.017266 18642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:26.018405 18642 hybrid_clock.cc:648] HybridClock initialized: now 1790303246018395 us; error 24 us; skew 500 ppm
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:26.020711 18642 init.cc:377] Logged in from keytab as oryx/127.11.220.195@KRBTEST.COM (short username oryx)
I20260925 02:27:26.021059 18642 webserver.cc:460] Webserver started at http://127.11.220.195:33709/ using document root <none> and password file <none>
I20260925 02:27:26.021247 18642 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:26.021287 18642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:26.021409 18642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:26.022281 18642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "3dcf8735aa5b4f259eddc6eee227592a"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "59a1a73acb462126e1a201b6ee7e48a0"
server_key_iv: "8a0cef787c66bdbc847dd21013356c02"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.022588 18642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "3dcf8735aa5b4f259eddc6eee227592a"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "59a1a73acb462126e1a201b6ee7e48a0"
server_key_iv: "8a0cef787c66bdbc847dd21013356c02"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.023782 18642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:26.024632 18658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.024799 18642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:26.024883 18642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "3dcf8735aa5b4f259eddc6eee227592a"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "59a1a73acb462126e1a201b6ee7e48a0"
server_key_iv: "8a0cef787c66bdbc847dd21013356c02"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.024948 18642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:26.035094 18642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:26.036870 18642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:26.037027 18642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:26.037240 18642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:26.037518 18642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:26.037566 18642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.037604 18642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:26.037631 18642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.042974 18642 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:39659
I20260925 02:27:26.043016 18772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:39659 every 8 connection(s)
I20260925 02:27:26.043283 18642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:26.048599 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 18642
I20260925 02:27:26.048705 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:26.048866 12147 external_mini_cluster.cc:1428] Setting key 738b8d10e16c0b0ccb882b9cc454628a
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:26.051928 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.043940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:40527 (local address 127.11.220.254:33109)
0925 02:27:26.043984 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.043987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.044198 (+ 211us) server_negotiation.cc:407] Connection header received
0925 02:27:26.044800 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.044802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.044814 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.044876 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.045758 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.046239 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.047183 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.047372 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.047447 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.049671 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.049673 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.049673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.049695 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.050849 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.051233 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.051234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.051234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.051281 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.051474 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.051474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.051475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.051565 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.051613 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.051806 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.051868 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 02:27:26.052116 18775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.043904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.195:40527)
0925 02:27:26.044073 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.044115 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.044673 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.044937 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.044940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.045475 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.045647 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.045651 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.046333 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.046335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.047091 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.047093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.047454 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.047456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.047577 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.048051 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.048060 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.049587 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.050953 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.050960 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.050962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.051173 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.051337 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.051337 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.051338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.051425 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.051616 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.051621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.051720 (+ 99us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.051793 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.051897 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260925 02:27:26.052537 18773 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33109
I20260925 02:27:26.052618 18773 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:26.052738 18773 heartbeater.cc:511] Master 127.11.220.254:33109 requested a full tablet report, sending...
I20260925 02:27:26.053134 18309 ts_manager.cc:194] Registered new tserver with Master: 3dcf8735aa5b4f259eddc6eee227592a (127.11.220.195:39659)
I20260925 02:27:26.053454 18309 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:40527
I20260925 02:27:26.060580 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.220.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.253.keytab.
Entry for principal oryx/127.11.220.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.253.keytab.
I20260925 02:27:26.082655 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.253:45953
--webserver_interface=127.11.220.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.253.keytab
--principal=oryx/127.11.220.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:33109,127.11.220.253:45953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:26.155551 18778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:26.155704 18778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:26.155721 18778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:26.156735 18778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:26.156770 18778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:26.156782 18778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:26.156795 18778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:26.156804 18778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:26.158272 18778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:33109,127.11.220.253:45953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.253.keytab
--principal=oryx/127.11.220.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.253:45953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.220.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
W20260925 02:27:26.158371 18778 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.220.254:33109,127.11.220.253:45953'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 02:27:26.158463 18778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:26.158607 18778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:26.162854 18778 server_base.cc:1034] running on GCE node
W20260925 02:27:26.162803 18786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.162822 18783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.162801 18784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.163203 18778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:26.163411 18778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:26.164580 18778 hybrid_clock.cc:648] HybridClock initialized: now 1790303246164567 us; error 48 us; skew 500 ppm
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:26.166756 18778 init.cc:377] Logged in from keytab as oryx/127.11.220.253@KRBTEST.COM (short username oryx)
I20260925 02:27:26.167181 18778 webserver.cc:460] Webserver started at http://127.11.220.253:46351/ using document root <none> and password file <none>
I20260925 02:27:26.167394 18778 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:26.167455 18778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:26.167579 18778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:26.168553 18778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data/instance:
uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "0c9974001b2e28013eae8b5665855100"
server_key_iv: "7c627c360e67c3076f5e9f53f3530b89"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.168867 18778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal/instance:
uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "0c9974001b2e28013eae8b5665855100"
server_key_iv: "7c627c360e67c3076f5e9f53f3530b89"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.170011 18778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:26.170601 18793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.170753 18778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:26.170837 18778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal
uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "0c9974001b2e28013eae8b5665855100"
server_key_iv: "7c627c360e67c3076f5e9f53f3530b89"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.170912 18778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:26.182994 18778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:26.184734 18778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:26.184867 18778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:26.188241 18778 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.253:45953
I20260925 02:27:26.188268 18846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.253:45953 every 8 connection(s)
I20260925 02:27:26.188622 18778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
I20260925 02:27:26.189641 18847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:26.191030 18847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.253@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:26.196372 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 18778
I20260925 02:27:26.198908 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.191639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37940 (local address 127.11.220.254:33109)
0925 02:27:26.191708 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.191710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.191843 (+ 133us) server_negotiation.cc:407] Connection header received
0925 02:27:26.192220 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.192222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.192231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.192278 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.192818 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.193297 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.194175 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.194340 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.194390 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.196054 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.196055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.196055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.196062 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.197630 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.198041 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.198042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.198042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.198091 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.198330 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.198330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.198331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.198448 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.198493 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.198761 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.198829 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:27:26.198977 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.191587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:37940)
0925 02:27:26.191746 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.191776 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.192149 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.192321 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.192323 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.192591 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.192756 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.192760 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.193365 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.193366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.194090 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.194093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.194406 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.194407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.194494 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.194835 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.194842 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.195978 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.197715 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.197721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.197723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.197972 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.198150 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.198151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.198151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.198275 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.198506 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.198510 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.198627 (+ 117us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.198739 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.198834 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260925 02:27:26.199478 18847 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.220.253@KRBTEST.COM
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.253@KRBTEST.COM for oryx/127.11.220.253@KRBTEST.COM
I20260925 02:27:26.204653 18851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.196813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.11.220.253:45953)
0925 02:27:26.196956 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.196961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.196977 (+ 16us) server_negotiation.cc:407] Connection header received
0925 02:27:26.197000 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.197004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.197019 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.197072 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.197714 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.198596 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.199490 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.199882 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.199954 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.202161 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.202167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.202168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.202173 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.203567 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.203877 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.203881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.203881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.203938 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.204123 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.204124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.204124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.204253 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.204384 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.204488 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.204556 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260925 02:27:26.205875 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.199662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:45953 (local address 127.0.0.1:51114)
0925 02:27:26.199704 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.199712 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.199740 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.199928 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.199928 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.200075 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.200110 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.200112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.200741 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.200742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.201176 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.201177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.201501 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.201502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.201587 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.201919 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.201925 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.202971 (+ 1046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.204993 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.204994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.204994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.205200 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.205376 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.205376 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.205377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.205422 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.205629 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.205629 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.205659 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.205713 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.205788 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 02:27:26.205948 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.199721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51114 (local address 127.11.220.253:45953)
0925 02:27:26.199844 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.199845 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.199849 (+ 4us) server_negotiation.cc:407] Connection header received
0925 02:27:26.199861 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.199861 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.199867 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.199926 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.200192 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.200674 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.201266 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.201436 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.201483 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.203064 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.203065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.203065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.203070 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.204918 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.205258 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.205259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.205259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.205314 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.205480 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.205480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.205481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.205573 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.205635 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.205728 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.205829 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260925 02:27:26.206986 18847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: Bootstrap starting.
I20260925 02:27:26.207476 18847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:26.207743 18847 log.cc:824] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:26.208324 18847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: No bootstrap required, opened a new log
I20260925 02:27:26.209504 18847 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } }
I20260925 02:27:26.209604 18847 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:26.209642 18847 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a1b01a39994e5da1e75e5f4f4fafb1, State: Initialized, Role: FOLLOWER
I20260925 02:27:26.209749 18847 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } }
I20260925 02:27:26.209955 18854 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } } }
I20260925 02:27:26.209991 18847 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:26.210044 18854 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:27:26.211341 18865 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:27:26.211413 18865 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:27:26.212759 18778 master_runner.cc:428] Detected that this master b5a1b01a39994e5da1e75e5f4f4fafb1 is joining an existing cluster
I20260925 02:27:26.212808 18778 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.220.253:45953
I20260925 02:27:26.219300 18871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.213768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:45953 (local address 127.0.0.1:51122)
0925 02:27:26.213941 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.213952 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.213999 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.214129 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.214130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.214352 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.214402 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.214403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.214970 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.214970 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.215415 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.215416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.215764 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.215765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.215862 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.216271 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.216275 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.216601 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.218348 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.218349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.218349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.218505 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.218683 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.218684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.218684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.218729 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.218962 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.218963 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.218995 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.219091 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.219176 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":74,"threads_started":1}
I20260925 02:27:26.219344 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.213793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51122 (local address 127.11.220.253:45953)
0925 02:27:26.213902 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.213903 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.213990 (+ 87us) server_negotiation.cc:407] Connection header received
0925 02:27:26.214044 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.214045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.214060 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.214126 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.214473 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.214888 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.215503 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.215673 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.215763 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.216696 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.216697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.216697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.216704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.218275 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.218575 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.218576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.218577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.218621 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.218793 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.218794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.218795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.218889 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.218958 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.219111 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.219223 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":94}
I20260925 02:27:26.220211 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.213862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.11.220.254:33109)
0925 02:27:26.213902 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.213904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.213941 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:27:26.214018 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.214019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.214028 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.214081 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.214522 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.215172 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.216174 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.216481 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.216521 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.217491 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.217492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.217493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.217499 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.218966 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.219308 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.219309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.219309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.219355 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.219762 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.219763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.219763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.219895 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.219946 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.220107 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.220156 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:27:26.220386 18872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.213727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:37950)
0925 02:27:26.213885 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.213895 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.213957 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.214119 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.214120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.214371 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.214430 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.214432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.215256 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.215257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.216074 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.216076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.216594 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.216595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.216661 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.217035 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.217040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.217422 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.219046 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.219047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.219047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.219232 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.219485 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.219486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.219486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.219681 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.219961 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.219962 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.220009 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.220091 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.220179 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.254@KRBTEST.COM for oryx/127.11.220.253@KRBTEST.COM
I20260925 02:27:26.228686 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.222377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:45953 (local address 127.0.0.1:51124)
0925 02:27:26.222464 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.222496 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.222560 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.222684 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.222686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.222869 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.222903 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.222904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.223630 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.223631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.224113 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.224114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.224564 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.224565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.224628 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.224987 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.224991 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.226232 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.227718 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.227724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.227725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.227887 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.228073 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.228074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.228074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.228129 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.228387 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.228393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.228438 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.228538 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.228620 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":48,"threads_started":1}
I20260925 02:27:26.228710 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.222431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51124 (local address 127.11.220.253:45953)
0925 02:27:26.222462 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.222463 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.222550 (+ 87us) server_negotiation.cc:407] Connection header received
0925 02:27:26.222608 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.222609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.222617 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.222682 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.222961 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.223560 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.224181 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.224487 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.224551 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.226327 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.226328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.226328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.226335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.227643 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.227951 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.227952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.227952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.228002 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.228179 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.228179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.228179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.228278 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.228333 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.228565 (+ 232us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.228630 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:27:26.229204 18309 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.220.253:45953: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1"
member_type: NON_VOTER
last_known_addr {
host: "127.11.220.253"
port: 45953
}
attrs {
promote: true
}
}
dest_uuid: "d0e02c0ed8aa4340be749d8f7b461dea"
cas_config_opid_index: -1
I20260925 02:27:26.229460 18309 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } }
I20260925 02:27:26.229915 18875 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b5a1b01a39994e5da1e75e5f4f4fafb1 (127.11.220.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } } }
I20260925 02:27:26.230094 18821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:26.230692 18821 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Refusing update from remote peer d0e02c0ed8aa4340be749d8f7b461dea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 02:27:26.230850 18875 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.220.253:45953
I20260925 02:27:26.230969 18874 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } } }
I20260925 02:27:26.231012 18778 master.cc:537] Master@127.11.220.253:45953 shutting down...
I20260925 02:27:26.231045 18874 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:26.231125 18874 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 02:27:26.231652 18296 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260925 02:27:26.232015 18296 consensus_peers.cc:487] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea -> Peer b5a1b01a39994e5da1e75e5f4f4fafb1 (127.11.220.253:45953): Couldn't send request to peer b5a1b01a39994e5da1e75e5f4f4fafb1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 02:27:26.233425 18778 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:26.233500 18778 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:26.233570 18778 tablet_replica.cc:330] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: stopping tablet replica
I20260925 02:27:26.245594 18778 master.cc:552] Master@127.11.220.253:45953 shutdown complete.
I20260925 02:27:26.245652 18778 master_runner.cc:305] Clearing existing system tablet
I20260925 02:27:26.245769 18778 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:26.245826 18778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:26.246411 18778 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:26.246922 18879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.247030 18778 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:26.247094 18778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.247130 18778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal
uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "0c9974001b2e28013eae8b5665855100"
server_key_iv: "7c627c360e67c3076f5e9f53f3530b89"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.247737 18778 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:26.248864 18778 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 02:27:26.248914 18778 log.cc:1196] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 02:27:26.249099 18778 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: Deleting consensus metadata
I20260925 02:27:26.249209 18778 master_runner.cc:315] Copying system tablet from 127.11.220.254:33109
I20260925 02:27:26.249737 18778 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: tablet copy: Beginning tablet copy session from remote peer at address 127.11.220.254:33109
I20260925 02:27:26.255244 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.250033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37966 (local address 127.11.220.254:33109)
0925 02:27:26.250081 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.250083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.250192 (+ 109us) server_negotiation.cc:407] Connection header received
0925 02:27:26.250259 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.250260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.250269 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.250321 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.250641 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.251080 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.251733 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.251916 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.251978 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.252848 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.252849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.252850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.252856 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.254187 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.254514 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.254515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.254516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.254577 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.254782 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.254783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.254783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.254883 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.254947 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.255101 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.255165 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 02:27:26.255270 18885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.249983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:37966)
0925 02:27:26.250128 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.250137 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.250206 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.250336 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.250337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.250502 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.250550 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.250552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.251153 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.251154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.251649 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.251650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.251987 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.251988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.252058 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.252501 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.252505 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.252750 (+ 245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.254270 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.254271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.254272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.254433 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.254653 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.254654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.254654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.254726 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.254949 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.254949 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.254988 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.255076 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.255167 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":59,"threads_started":1}
I20260925 02:27:26.255489 18329 tablet_copy_service.cc:140] P d0e02c0ed8aa4340be749d8f7b461dea: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b5a1b01a39994e5da1e75e5f4f4fafb1 ({username='oryx', principal='oryx/127.11.220.253@KRBTEST.COM'} at 127.0.0.1:37966)
I20260925 02:27:26.255563 18329 tablet_copy_service.cc:161] P d0e02c0ed8aa4340be749d8f7b461dea: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b5a1b01a39994e5da1e75e5f4f4fafb1 at {username='oryx', principal='oryx/127.11.220.253@KRBTEST.COM'} at 127.0.0.1:37966: session id = b5a1b01a39994e5da1e75e5f4f4fafb1-00000000000000000000000000000000
I20260925 02:27:26.256301 18329 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 02:27:26.256896 18778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 02:27:26.257759 18778 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: tablet copy: Starting download of 0 data blocks...
I20260925 02:27:26.257897 18778 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: tablet copy: Starting download of 1 WAL segments...
I20260925 02:27:26.258622 18778 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 02:27:26.259315 18329 tablet_copy_service.cc:342] P d0e02c0ed8aa4340be749d8f7b461dea: Request end of tablet copy session b5a1b01a39994e5da1e75e5f4f4fafb1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.220.253@KRBTEST.COM'} at 127.0.0.1:37966
I20260925 02:27:26.259370 18329 tablet_copy_service.cc:433] P d0e02c0ed8aa4340be749d8f7b461dea: ending tablet copy session b5a1b01a39994e5da1e75e5f4f4fafb1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b5a1b01a39994e5da1e75e5f4f4fafb1
I20260925 02:27:26.261548 18778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:26.262598 18887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.262656 18886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.262744 18889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.262843 18778 server_base.cc:1034] running on GCE node
I20260925 02:27:26.262915 18778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:26.263029 18778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:26.264155 18778 hybrid_clock.cc:648] HybridClock initialized: now 1790303246264142 us; error 27 us; skew 500 ppm
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:26.265328 18778 init.cc:377] Logged in from keytab as oryx/127.11.220.253@KRBTEST.COM (short username oryx)
I20260925 02:27:26.265549 18778 webserver.cc:460] Webserver started at http://127.11.220.253:45153/ using document root <none> and password file <none>
I20260925 02:27:26.265663 18778 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:26.265712 18778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:26.266278 18778 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:26.266753 18896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.266870 18778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.266927 18778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal
uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "0c9974001b2e28013eae8b5665855100"
server_key_iv: "7c627c360e67c3076f5e9f53f3530b89"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.266969 18778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:26.276572 18778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:26.277828 18778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:26.280650 18778 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.253:45953
I20260925 02:27:26.280867 18949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.253:45953 every 8 connection(s)
I20260925 02:27:26.280966 18778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
I20260925 02:27:26.281718 18950 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:27:26.282209 18950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: Bootstrap starting.
I20260925 02:27:26.284963 18950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:27:26.285180 18950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: Bootstrap complete.
I20260925 02:27:26.285358 18950 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } }
I20260925 02:27:26.285444 18950 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b5a1b01a39994e5da1e75e5f4f4fafb1, State: Initialized, Role: LEARNER
I20260925 02:27:26.285478 18950 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } }
I20260925 02:27:26.285687 18950 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:26.285665 18953 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } } }
I20260925 02:27:26.285746 18953 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: This master's current role is: LEARNER
I20260925 02:27:26.287137 18965 catalog_manager.cc:1259] Loaded cluster ID: 013950402d7b4ece8b5dc4d4425796f1
I20260925 02:27:26.287177 18965 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: loading cluster ID for follower catalog manager: success
I20260925 02:27:26.288168 18965 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: acquiring CA information for follower catalog manager: success
I20260925 02:27:26.288514 18965 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:27:26.312933 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.306322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.11.220.253:45953)
0925 02:27:26.306439 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.306441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.306450 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:26.306473 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.306474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.306484 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.306567 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.307014 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.308067 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.308742 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.309051 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.309123 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.310200 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.310202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.310202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.310209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.311891 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.312205 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.312206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.312206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.312269 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.312435 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.312436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.312436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.312569 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.312675 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.312746 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.312820 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
WARNING: no policy specified for oryx/127.11.220.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.252.keytab.
Entry for principal oryx/127.11.220.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.252.keytab.
I20260925 02:27:26.334873 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.252:44691
--webserver_interface=127.11.220.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.252.keytab
--principal=oryx/127.11.220.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:33109,127.11.220.253:45953,127.11.220.252:44691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:26.409370 18969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:26.409540 18969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:26.409559 18969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:26.410584 18969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:26.410632 18969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:26.410646 18969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:26.410657 18969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:26.410686 18969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:26.412056 18969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:33109,127.11.220.253:45953,127.11.220.252:44691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.252.keytab
--principal=oryx/127.11.220.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.252:44691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.220.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:26.412238 18969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:26.412370 18969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:26.416644 18975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.416692 18974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.416658 18977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.416884 18969 server_base.cc:1034] running on GCE node
I20260925 02:27:26.417135 18969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:26.417376 18969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:26.418530 18969 hybrid_clock.cc:648] HybridClock initialized: now 1790303246418531 us; error 41 us; skew 500 ppm
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:26.420802 18969 init.cc:377] Logged in from keytab as oryx/127.11.220.252@KRBTEST.COM (short username oryx)
I20260925 02:27:26.421175 18969 webserver.cc:460] Webserver started at http://127.11.220.252:39947/ using document root <none> and password file <none>
I20260925 02:27:26.421382 18969 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:26.421451 18969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:26.421617 18969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:26.422574 18969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data/instance:
uuid: "340bae31edf543e6bebaf5a40119ba4b"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "9095f6134d5941aeede7d57157fca10f"
server_key_iv: "0b1164ea026dc88813c0cbfa3adfcd0f"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.422923 18969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal/instance:
uuid: "340bae31edf543e6bebaf5a40119ba4b"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "9095f6134d5941aeede7d57157fca10f"
server_key_iv: "0b1164ea026dc88813c0cbfa3adfcd0f"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.424086 18969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 02:27:26.424763 18985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.424911 18969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.425007 18969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal
uuid: "340bae31edf543e6bebaf5a40119ba4b"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "9095f6134d5941aeede7d57157fca10f"
server_key_iv: "0b1164ea026dc88813c0cbfa3adfcd0f"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.425073 18969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:26.439343 18969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:26.440892 18969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:26.441027 18969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:26.444761 18969 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.252:44691
I20260925 02:27:26.444787 19037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.252:44691 every 8 connection(s)
I20260925 02:27:26.445065 18969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
I20260925 02:27:26.445900 19038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:26.447212 19038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:27:26.448827 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 18969
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.252@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.220.252@KRBTEST.COM
I20260925 02:27:26.456241 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.447901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57384 (local address 127.11.220.254:33109)
0925 02:27:26.447956 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.447958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.448022 (+ 64us) server_negotiation.cc:407] Connection header received
0925 02:27:26.448415 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.448416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.448424 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.448514 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.449092 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.449839 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.451115 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.451323 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.451406 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.453663 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.453664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.453665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.453672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.455060 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.455418 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.455419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.455419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.455475 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.455695 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.455696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.455696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.455801 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.455869 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.456122 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.456184 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 02:27:26.456429 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.447764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57384)
0925 02:27:26.447907 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.447942 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.448340 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.448577 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.448580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.448786 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.448994 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.448999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.449931 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.449931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.451035 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.451038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.451428 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.451429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.451516 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.451998 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.452006 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.453573 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.455155 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.455161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.455163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.455348 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.455542 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.455543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.455543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.455636 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.455872 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.455879 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.455984 (+ 105us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.456092 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.456227 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260925 02:27:26.456954 19038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:27:26.457445 19041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.449281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32770 (local address 127.11.220.252:44691)
0925 02:27:26.449439 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.449442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.449456 (+ 14us) server_negotiation.cc:407] Connection header received
0925 02:27:26.449473 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.449474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.449493 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.449550 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.450184 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.451205 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.451953 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.452232 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.452307 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.454654 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.454660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.454661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.454667 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.455961 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.456412 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.456414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.456415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.456660 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.456910 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.456911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.456911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.457038 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.457170 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.457262 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.457333 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":126,"thread_start_us":103,"threads_started":1}
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.252@KRBTEST.COM for oryx/127.11.220.253@KRBTEST.COM
I20260925 02:27:26.462349 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.457156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:45953 (local address 127.0.0.1:50096)
0925 02:27:26.457192 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.457199 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.457225 (+ 26us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.457367 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.457368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.457507 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.457543 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.457545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.458286 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.458287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.458714 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.458715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.458977 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.458977 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.459010 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.459293 (+ 283us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.459297 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.460246 (+ 949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.461704 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.461704 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.461704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.461832 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.461986 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.461987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.461987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.462025 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.462158 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.462158 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.462184 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.462239 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.462303 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 02:27:26.462373 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.457215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50096 (local address 127.11.220.253:45953)
0925 02:27:26.457253 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.457255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.457261 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:27:26.457286 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.457286 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.457298 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.457351 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.457638 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.458225 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.458771 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.458927 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.458966 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.460319 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.460320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.460320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.460326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.461647 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.461898 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.461899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.461899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.461939 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.462057 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.462057 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.462057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.462129 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.462166 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.462266 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.462320 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:27:26.462769 19038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.252@KRBTEST.COM for oryx/127.11.220.252@KRBTEST.COM
I20260925 02:27:26.468068 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.462987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:44691 (local address 127.0.0.1:32778)
0925 02:27:26.463032 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.463042 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.463078 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.463177 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.463178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.463267 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.463289 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.463290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.463830 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.463831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.464245 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.464246 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.464605 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.464605 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.464637 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.464900 (+ 263us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.464903 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.465867 (+ 964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.467273 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.467274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.467274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.467416 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.467564 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.467565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.467565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.467607 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.467828 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.467828 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.467897 (+ 69us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.467967 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.468021 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 02:27:26.468211 19041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.463027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32778 (local address 127.11.220.252:44691)
0925 02:27:26.463058 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.463060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.463070 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:27:26.463130 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.463130 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.463136 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.463167 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.463343 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.463765 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.464313 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.464550 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.464594 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.465959 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.465960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.465960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.465966 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.467216 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.467466 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.467466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.467466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.467501 (+ 35us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.467662 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.467663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.467663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.467752 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.467796 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.468009 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.468064 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 02:27:26.469244 19038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Bootstrap starting.
I20260925 02:27:26.469985 19038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:26.470405 19038 log.cc:824] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:26.470969 19038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: No bootstrap required, opened a new log
I20260925 02:27:26.472194 19038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } }
I20260925 02:27:26.472292 19038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:26.472335 19038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 340bae31edf543e6bebaf5a40119ba4b, State: Initialized, Role: FOLLOWER
I20260925 02:27:26.472476 19038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } }
I20260925 02:27:26.472689 19043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } } }
I20260925 02:27:26.472731 19038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:26.472779 19043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:27:26.474215 19054 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:27:26.474267 19054 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:27:26.476245 18969 master_runner.cc:428] Detected that this master 340bae31edf543e6bebaf5a40119ba4b is joining an existing cluster
I20260925 02:27:26.476287 18969 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.220.252:44691
I20260925 02:27:26.482928 19041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.477504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32780 (local address 127.11.220.252:44691)
0925 02:27:26.477533 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.477534 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.477573 (+ 39us) server_negotiation.cc:407] Connection header received
0925 02:27:26.477650 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.477650 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.477658 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.477683 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.478074 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.478701 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.479590 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.479818 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.479848 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.480820 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.480821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.480821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.480827 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.481992 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.482292 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.482293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.482293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.482333 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.482488 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.482488 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.482489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.482577 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.482650 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.482781 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.482847 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":15}
I20260925 02:27:26.483006 19061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.477391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:44691 (local address 127.0.0.1:32780)
0925 02:27:26.477518 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.477528 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.477591 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.477729 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.477730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.477948 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.478004 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.478006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.478771 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.478771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.479517 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.479519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.479911 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.479911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.479959 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.480374 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.480378 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.480710 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.482078 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.482079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.482079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.482223 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.482393 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.482393 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.482394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.482438 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.482646 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.482646 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.482689 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.482763 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.482891 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":56,"thread_start_us":54,"threads_started":1}
I20260925 02:27:26.504006 19062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.477194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57394)
0925 02:27:26.477553 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.477560 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.477633 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.477789 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.477790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.478004 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.478060 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.478062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.478859 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.478860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.479626 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.479627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.480082 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.480082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.480139 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.500985 (+ 20846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.500989 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.501334 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.502977 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.502978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.502978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.503223 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.503403 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.503404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.503404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.503457 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.503719 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.503720 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.503759 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.503825 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.503909 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":40,"thread_start_us":78,"threads_started":1}
I20260925 02:27:26.504014 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.477253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57394 (local address 127.11.220.254:33109)
0925 02:27:26.477298 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.477300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.477610 (+ 310us) server_negotiation.cc:407] Connection header received
0925 02:27:26.477698 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.477699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.477709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.477737 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.478130 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.478783 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.479695 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.479957 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.479993 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.501434 (+ 21441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.501435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.501436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.501442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.502885 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.503295 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.503296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.503296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.503351 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.503527 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.503528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.503528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.503651 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.503716 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.503862 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.503934 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 02:27:26.504062 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.477332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50112 (local address 127.11.220.253:45953)
0925 02:27:26.477376 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.477378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.477478 (+ 100us) server_negotiation.cc:407] Connection header received
0925 02:27:26.477578 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.477579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.477589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.477619 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.478067 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.478724 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.479763 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.480045 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.480083 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.501440 (+ 21357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.501442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.501442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.501447 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.502991 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.503348 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.503349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.503350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.503400 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.503563 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.503564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.503564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.503668 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.503721 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.503914 (+ 193us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.503979 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:27:26.504423 19060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.477194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:45953 (local address 127.0.0.1:50112)
0925 02:27:26.477409 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.477419 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.477506 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.477690 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.477691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.477934 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.477992 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.477993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.478813 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.478813 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.479672 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.479674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.480150 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.480150 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.480206 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.500915 (+ 20709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.500920 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.501352 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.503063 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.503063 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.503064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.503285 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.503452 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.503453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.503453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.503509 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.503747 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.503747 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.503790 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.504213 (+ 423us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.504327 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":27,"thread_start_us":77,"threads_started":1}
I20260925 02:27:26.508534 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 18371
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303245, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.254@KRBTEST.COM for oryx/127.11.220.252@KRBTEST.COM
I20260925 02:27:26.511642 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.505929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:44691 (local address 127.0.0.1:32788)
0925 02:27:26.505974 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.505983 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.506018 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.506142 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.506143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.506273 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.506303 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.506304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.506827 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.506828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.507284 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.507285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.507623 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.507624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.507686 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.508042 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.508046 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.509288 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.510705 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.510705 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.510706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.510856 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.511047 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.511048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.511049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.511099 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.511351 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.511352 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.511391 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.511466 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.511560 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 02:27:26.511641 19041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.505983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.11.220.252:44691)
0925 02:27:26.506018 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.506020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.506028 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:26.506075 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.506076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.506083 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.506138 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.506370 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.506757 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.507371 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.507543 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.507617 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.509375 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.509377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.509377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.509384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.510613 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.510924 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.510925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.510926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.510971 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.511171 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.511172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.511172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.511290 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.511357 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.511499 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.511557 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:26.513818 18309 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.220.252:44691: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b"
member_type: NON_VOTER
last_known_addr {
host: "127.11.220.252"
port: 44691
}
attrs {
promote: true
}
}
dest_uuid: "d0e02c0ed8aa4340be749d8f7b461dea"
cas_config_opid_index: 5
I20260925 02:27:26.514051 18309 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } }
I20260925 02:27:26.514488 18875 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 340bae31edf543e6bebaf5a40119ba4b (127.11.220.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.515377 18875 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.220.252:44691
I20260925 02:27:26.515442 18875 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.515504 18875 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:26.515951 19012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:26.516784 18969 master.cc:537] Master@127.11.220.252:44691 shutting down...
I20260925 02:27:26.516812 19012 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 FOLLOWER]: Refusing update from remote peer d0e02c0ed8aa4340be749d8f7b461dea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 02:27:26.517078 18874 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Connected to new peer: Peer: permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 02:27:26.517815 18293 consensus_peers.cc:487] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea -> Peer 340bae31edf543e6bebaf5a40119ba4b (127.11.220.252:44691): Couldn't send request to peer 340bae31edf543e6bebaf5a40119ba4b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 02:27:26.518450 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:43443
--local_ip_for_outbound_sockets=127.11.220.193
--tserver_master_addrs=127.11.220.254:33109
--webserver_port=41025
--webserver_interface=127.11.220.193
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab
--principal=oryx/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.220.254:33109,127.11.220.253:45953
--tserver_master_addrs=127.11.220.254:33109,127.11.220.253:45953,127.11.220.252:44691 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:27:26.520412 18969 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:26.520521 18969 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:26.520576 18969 tablet_replica.cc:330] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: stopping tablet replica
I20260925 02:27:26.522970 18969 master.cc:552] Master@127.11.220.252:44691 shutdown complete.
I20260925 02:27:26.523018 18969 master_runner.cc:305] Clearing existing system tablet
I20260925 02:27:26.523147 18969 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:26.523173 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.515646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:45953 (local address 127.0.0.1:50118)
0925 02:27:26.516842 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.516852 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.517943 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.518165 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.518167 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.518168 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:26.518207 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.518208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.520601 (+ 2393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.520602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.522137 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.522140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.522951 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.522952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.523013 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.523032 (+ 19us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.523079 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.523111 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1180}
I20260925 02:27:26.523200 18969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:26.523396 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.515711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.11.220.253:45953)
0925 02:27:26.516794 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.516813 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.517917 (+ 1104us) server_negotiation.cc:407] Connection header received
0925 02:27:26.518027 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.518028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.518052 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.518139 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:26.519846 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.520513 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.522221 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.522797 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.522862 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.522918 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.523296 (+ 378us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.523336 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1064}
I20260925 02:27:26.523571 18923 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 LEARNER]: Refusing update from remote peer d0e02c0ed8aa4340be749d8f7b461dea: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 02:27:26.523759 18874 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 02:27:26.523874 18969 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:26.524322 19067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.524251 18923 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 340bae31edf543e6bebaf5a40119ba4b (127.11.220.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.524420 18969 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:26.524484 18969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.524513 18969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal
uuid: "340bae31edf543e6bebaf5a40119ba4b"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "9095f6134d5941aeede7d57157fca10f"
server_key_iv: "0b1164ea026dc88813c0cbfa3adfcd0f"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.525347 18969 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:26.525439 18953 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0e02c0ed8aa4340be749d8f7b461dea. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.525523 18953 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: This master's current role is: LEARNER
I20260925 02:27:26.525558 18953 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: NON_VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: true } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.525597 18953 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: This master's current role is: LEARNER
I20260925 02:27:26.526010 18874 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea: attempting to promote NON_VOTER b5a1b01a39994e5da1e75e5f4f4fafb1 to VOTER
I20260925 02:27:26.526180 18874 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } }
I20260925 02:27:26.526640 18923 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 LEARNER]: Refusing update from remote peer d0e02c0ed8aa4340be749d8f7b461dea: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 02:27:26.526721 18969 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 02:27:26.526787 18969 log.cc:1196] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 02:27:26.526803 18875 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 02:27:26.527001 18969 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Deleting consensus metadata
I20260925 02:27:26.527086 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.526511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:44691 (local address 127.0.0.1:32804)
0925 02:27:26.527025 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.527064 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:44691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502}
I20260925 02:27:26.527125 18969 master_runner.cc:315] Copying system tablet from 127.11.220.254:33109
I20260925 02:27:26.527423 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.527264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:44691 (local address 127.0.0.1:32820)
0925 02:27:26.527381 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.527403 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:44691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260925 02:27:26.527483 18293 consensus_peers.cc:487] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea -> Peer 340bae31edf543e6bebaf5a40119ba4b (127.11.220.252:44691): Couldn't send request to peer 340bae31edf543e6bebaf5a40119ba4b. Status: Network error: Client connection negotiation failed: client connection to 127.11.220.252:44691: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 02:27:26.527693 18969 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: tablet copy: Beginning tablet copy session from remote peer at address 127.11.220.254:33109
I20260925 02:27:26.527763 18875 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b5a1b01a39994e5da1e75e5f4f4fafb1 (127.11.220.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.528614 18875 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.528707 18875 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:26.528959 18874 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.529022 18874 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:26.529241 18923 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b5a1b01a39994e5da1e75e5f4f4fafb1 (127.11.220.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.530117 19068 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.530202 19068 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:27:26.534515 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.528162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57396 (local address 127.11.220.254:33109)
0925 02:27:26.528192 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.528193 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.528371 (+ 178us) server_negotiation.cc:407] Connection header received
0925 02:27:26.528491 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.528493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.528502 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.528533 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.528941 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.529674 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.530669 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.530958 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.530998 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.532003 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.532004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.532004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.532011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.533454 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.533832 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.533833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.533834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.533880 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.534053 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.534054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.534054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.534158 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.534196 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.534374 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.534437 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:27:26.534811 19075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.528225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57396)
0925 02:27:26.528297 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.528306 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.528396 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.528616 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.528617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.528791 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.528849 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.528850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.529770 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.529771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.530559 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.530562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.531071 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.531072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.531132 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.531531 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.531536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.531903 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.533555 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.533556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.533557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.533755 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.533944 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.533945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.533945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.533992 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.534238 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.534239 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.534277 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.534625 (+ 348us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.534730 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"thread_start_us":36,"threads_started":1}
I20260925 02:27:26.535005 18329 tablet_copy_service.cc:140] P d0e02c0ed8aa4340be749d8f7b461dea: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 340bae31edf543e6bebaf5a40119ba4b ({username='oryx', principal='oryx/127.11.220.252@KRBTEST.COM'} at 127.0.0.1:57396)
I20260925 02:27:26.535053 18329 tablet_copy_service.cc:161] P d0e02c0ed8aa4340be749d8f7b461dea: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 340bae31edf543e6bebaf5a40119ba4b at {username='oryx', principal='oryx/127.11.220.252@KRBTEST.COM'} at 127.0.0.1:57396: session id = 340bae31edf543e6bebaf5a40119ba4b-00000000000000000000000000000000
I20260925 02:27:26.535535 18329 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 02:27:26.535928 18969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 02:27:26.536964 18969 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: tablet copy: Starting download of 0 data blocks...
I20260925 02:27:26.537102 18969 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: tablet copy: Starting download of 1 WAL segments...
I20260925 02:27:26.537595 18969 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 02:27:26.538276 18329 tablet_copy_service.cc:342] P d0e02c0ed8aa4340be749d8f7b461dea: Request end of tablet copy session 340bae31edf543e6bebaf5a40119ba4b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.220.252@KRBTEST.COM'} at 127.0.0.1:57396
I20260925 02:27:26.538323 18329 tablet_copy_service.cc:433] P d0e02c0ed8aa4340be749d8f7b461dea: ending tablet copy session 340bae31edf543e6bebaf5a40119ba4b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 340bae31edf543e6bebaf5a40119ba4b
I20260925 02:27:26.540879 18969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:26.541841 19077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.541919 19078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.544607 18969 server_base.cc:1034] running on GCE node
W20260925 02:27:26.544596 19081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.544806 18969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:26.544926 18969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:26.546051 18969 hybrid_clock.cc:648] HybridClock initialized: now 1790303246546048 us; error 49 us; skew 500 ppm
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:26.547322 18969 init.cc:377] Logged in from keytab as oryx/127.11.220.252@KRBTEST.COM (short username oryx)
I20260925 02:27:26.547545 18969 webserver.cc:460] Webserver started at http://127.11.220.252:33595/ using document root <none> and password file <none>
I20260925 02:27:26.547652 18969 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:26.547694 18969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:26.548352 18969 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:26.548921 19088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.549019 18969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.549050 18969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal
uuid: "340bae31edf543e6bebaf5a40119ba4b"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "9095f6134d5941aeede7d57157fca10f"
server_key_iv: "0b1164ea026dc88813c0cbfa3adfcd0f"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.549093 18969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:26.565348 18969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:26.567158 18969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:26.570300 18969 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.252:44691
I20260925 02:27:26.570629 18969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
I20260925 02:27:26.572614 19140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.252:44691 every 8 connection(s)
I20260925 02:27:26.573253 19141 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:27:26.573712 19141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Bootstrap starting.
I20260925 02:27:26.576373 19141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:27:26.576584 19141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Bootstrap complete.
I20260925 02:27:26.576804 19141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } }
I20260925 02:27:26.576884 19141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 340bae31edf543e6bebaf5a40119ba4b, State: Initialized, Role: LEARNER
I20260925 02:27:26.576913 19141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } }
I20260925 02:27:26.577088 19141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:26.577107 19144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.577301 19144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: This master's current role is: LEARNER
I20260925 02:27:26.578491 19155 catalog_manager.cc:1259] Loaded cluster ID: 013950402d7b4ece8b5dc4d4425796f1
I20260925 02:27:26.578537 19155 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: loading cluster ID for follower catalog manager: success
I20260925 02:27:26.579828 19155 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: acquiring CA information for follower catalog manager: success
I20260925 02:27:26.580133 19155 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 02:27:26.614944 19064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:26.615109 19064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:26.615128 19064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:26.616143 19064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:26.616176 19064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:26.616201 19064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:26.617540 19064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab
--principal=oryx/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:43443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=41025
--tserver_master_addrs=127.11.220.254:33109,127.11.220.253:45953,127.11.220.252:44691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:26.617691 19064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:26.617853 19064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:26.622344 19161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.622356 19160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.622349 19163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.622493 19064 server_base.cc:1034] running on GCE node
I20260925 02:27:26.622733 19064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:26.622958 19064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:26.624085 19064 hybrid_clock.cc:648] HybridClock initialized: now 1790303246624064 us; error 36 us; skew 500 ppm
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:26.626209 19064 init.cc:377] Logged in from keytab as oryx/127.11.220.193@KRBTEST.COM (short username oryx)
I20260925 02:27:26.626598 19064 webserver.cc:460] Webserver started at http://127.11.220.193:41025/ using document root <none> and password file <none>
I20260925 02:27:26.626813 19064 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:26.626905 19064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:26.628057 19064 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:26.628688 19170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.628837 19064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:26.628916 19064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "c4236f9f8b9f42de9809d168a1ee5bf6"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "acf0ac71024e80a0a75ec25165a41421"
server_key_iv: "8fa0f70381c08e70dc74587e406968a1"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.629186 19064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:26.643039 19064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:26.644641 19064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:26.644784 19064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:26.644979 19064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:26.645247 19064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:26.645293 19064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.645323 19064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:26.645346 19064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.650732 19064 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:43443
I20260925 02:27:26.650794 19284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:43443 every 8 connection(s)
I20260925 02:27:26.651093 19064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:26.653935 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 19064
I20260925 02:27:26.654078 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 18506
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.252@KRBTEST.COM
I20260925 02:27:26.660710 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:46147
--local_ip_for_outbound_sockets=127.11.220.194
--tserver_master_addrs=127.11.220.254:33109
--webserver_port=34879
--webserver_interface=127.11.220.194
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab
--principal=oryx/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.220.254:33109,127.11.220.253:45953
--tserver_master_addrs=127.11.220.254:33109,127.11.220.253:45953,127.11.220.252:44691 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:27:26.661832 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.651934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:44691 (local address 127.11.220.193:45229)
0925 02:27:26.652238 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.652245 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.652902 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.653293 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.653293 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.653533 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.653722 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.653728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.655310 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.655312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.656316 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.656319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.656851 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.656852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.656919 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.657343 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.657352 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.658720 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.660325 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.660331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.660333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.660645 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.660853 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.660854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.660854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.660966 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.661219 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.661224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.661361 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.661455 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.661598 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":54,"thread_start_us":53,"threads_started":1}
I20260925 02:27:26.662124 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.651994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:45229 (local address 127.11.220.252:44691)
0925 02:27:26.652110 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.652112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.652299 (+ 187us) server_negotiation.cc:407] Connection header received
0925 02:27:26.653191 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.653192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.653204 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.653272 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.653827 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.655230 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.656416 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.656762 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.656828 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.658808 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.658810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.658810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.658817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.660249 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.660728 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.660730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.660730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.660788 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.661038 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.661039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.661040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.661149 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.661198 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.661954 (+ 756us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.662025 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":50,"threads_started":1}
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:26.662442 19285 heartbeater.cc:347] Connected to a master server at 127.11.220.252:44691
I20260925 02:27:26.662510 19285 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:26.662663 19285 heartbeater.cc:511] Master 127.11.220.252:44691 requested a full tablet report, sending...
I20260925 02:27:26.663760 19098 ts_manager.cc:194] Registered new tserver with Master: c4236f9f8b9f42de9809d168a1ee5bf6 (127.11.220.193:43443)
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.253@KRBTEST.COM
I20260925 02:27:26.665576 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.651795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:51917 (local address 127.11.220.254:33109)
0925 02:27:26.651841 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.651843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.652005 (+ 162us) server_negotiation.cc:407] Connection header received
0925 02:27:26.652987 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.652988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.653001 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.653057 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.653827 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.654597 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.655768 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.655983 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.656053 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.662718 (+ 6665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.662720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.662720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.662727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.664301 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.664744 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.664745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.664746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.664804 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.665025 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.665026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.665027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.665177 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.665248 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.665404 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.665480 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 02:27:26.665767 19289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.651678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.193:51917)
0925 02:27:26.651899 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.651936 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.652889 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.653115 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.653118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.653500 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.653722 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.653728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.654740 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.654741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.655671 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.655673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.656060 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.656061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.656199 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.661369 (+ 5170us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.661373 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.662628 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.664413 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.664414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.664415 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.664660 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.664877 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.664878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.664878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.664935 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.665254 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.665255 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.665291 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.665364 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.665684 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":4,"thread_start_us":89,"threads_started":1}
I20260925 02:27:26.666354 19286 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33109
I20260925 02:27:26.666400 19286 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:26.666465 19286 heartbeater.cc:511] Master 127.11.220.254:33109 requested a full tablet report, sending...
I20260925 02:27:26.666715 18309 ts_manager.cc:194] Re-registered known tserver with Master: c4236f9f8b9f42de9809d168a1ee5bf6 (127.11.220.193:43443)
I20260925 02:27:26.667194 18309 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:51917
I20260925 02:27:26.667433 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.651696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:52851 (local address 127.11.220.253:45953)
0925 02:27:26.651745 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.651748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.652187 (+ 439us) server_negotiation.cc:407] Connection header received
0925 02:27:26.653057 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.653058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.653076 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.653138 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.660549 (+ 7411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.661250 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.662450 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.662724 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.662781 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.664659 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.664660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.664661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.664667 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.666252 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.666623 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.666624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.666624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.666680 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.666868 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.666869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.666869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.666996 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.667048 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.667255 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.667339 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 02:27:26.667578 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.651678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:45953 (local address 127.11.220.193:52851)
0925 02:27:26.652120 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.652128 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.652902 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.653158 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.653159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.653500 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.653722 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.653726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.661325 (+ 7599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.661326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.662313 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.662315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.662831 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.662832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.662890 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.663336 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.663341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.664567 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.666323 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.666324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.666324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.666533 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.666743 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.666744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.666745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.666795 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.667065 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.667066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.667110 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.667185 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.667275 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":18,"thread_start_us":82,"threads_started":1}
I20260925 02:27:26.667896 19287 heartbeater.cc:347] Connected to a master server at 127.11.220.253:45953
I20260925 02:27:26.667937 19287 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:26.668318 19287 heartbeater.cc:511] Master 127.11.220.253:45953 requested a full tablet report, sending...
I20260925 02:27:26.668668 18913 ts_manager.cc:194] Registered new tserver with Master: c4236f9f8b9f42de9809d168a1ee5bf6 (127.11.220.193:43443)
W20260925 02:27:26.735719 19293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:26.735905 19293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:26.735940 19293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:26.737015 19293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:26.737068 19293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:26.737109 19293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:26.738421 19293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab
--principal=oryx/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:46147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=34879
--tserver_master_addrs=127.11.220.254:33109,127.11.220.253:45953,127.11.220.252:44691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:26.738643 19293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:26.738842 19293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:26.743203 19301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.743227 19298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.743289 19293 server_base.cc:1034] running on GCE node
W20260925 02:27:26.743251 19299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.743507 19293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:26.743717 19293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:26.744859 19293 hybrid_clock.cc:648] HybridClock initialized: now 1790303246744850 us; error 28 us; skew 500 ppm
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:26.747076 19293 init.cc:377] Logged in from keytab as oryx/127.11.220.194@KRBTEST.COM (short username oryx)
I20260925 02:27:26.747426 19293 webserver.cc:460] Webserver started at http://127.11.220.194:34879/ using document root <none> and password file <none>
I20260925 02:27:26.747579 19293 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:26.747629 19293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:26.748838 19293 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:26.749397 19308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.749558 19293 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:26.749651 19293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "210baf669be6468a818f21f2e0f17139"
format_stamp: "Formatted at 2026-09-25 02:27:25 on dist-test-slave-dn4n"
server_key: "e5cb8602f9bd3435100edfbd92dd82af"
server_key_iv: "82d36b921ab4d726e25e2a102ea3d1eb"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.749943 19293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:26.760409 19293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:26.761858 19293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:26.761976 19293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:26.762157 19293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:26.762403 19293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:26.762435 19293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.762451 19293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:26.762488 19293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.767745 19293 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:46147
I20260925 02:27:26.767876 19425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:46147 every 8 connection(s)
I20260925 02:27:26.768132 19293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.252@KRBTEST.COM
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.253@KRBTEST.COM
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:26.776818 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 19293
I20260925 02:27:26.776973 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 18642
I20260925 02:27:26.779066 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.768733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:53643 (local address 127.11.220.252:44691)
0925 02:27:26.768775 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.768777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.768996 (+ 219us) server_negotiation.cc:407] Connection header received
0925 02:27:26.769941 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.769942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.769958 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.769985 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.770625 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.771253 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.772018 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.772296 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.772352 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.774506 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.774507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.774507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.774513 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.776024 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.776501 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.776503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.776503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.776557 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.778380 (+ 1823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.778381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.778381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.778498 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.778539 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.778886 (+ 347us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.778966 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:27:26.779346 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.769086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:44705 (local address 127.11.220.254:33109)
0925 02:27:26.769131 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.769133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.769142 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:26.769929 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.769930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.769942 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.769993 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.770609 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.771032 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.772160 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.772419 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.772526 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.775469 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.775471 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.775471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.775478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.777031 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.778591 (+ 1560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.778592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.778593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.778640 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.778846 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.778847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.778848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.778965 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.779004 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.779192 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.779269 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 02:27:26.779757 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.768739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:44691 (local address 127.11.220.194:53643)
0925 02:27:26.768887 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.768923 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.769836 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.770052 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.770055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.770386 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.770520 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.770525 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.771337 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.771338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.771913 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.771914 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.772365 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.772366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.772497 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.773016 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.773023 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.774425 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.776101 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.776107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.776109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.776386 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.776633 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.776633 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.776633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.778299 (+ 1666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.778581 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.778587 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.778720 (+ 133us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.779329 (+ 609us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.779630 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":25,"thread_start_us":87,"threads_started":1}
I20260925 02:27:26.779759 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.768980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.194:44705)
0925 02:27:26.769072 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.769083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.769847 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.770051 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.770055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.770360 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.770518 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.770522 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.771130 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.771131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.772075 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.772077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.772526 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.772527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.772575 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.773262 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.773266 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.775362 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.777104 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.777105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.777105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.778518 (+ 1413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.778711 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.778712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.778712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.778779 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.779047 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.779048 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.779083 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.779419 (+ 336us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.779546 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260925 02:27:26.780375 19427 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33109
I20260925 02:27:26.780483 19427 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:26.780638 19427 heartbeater.cc:511] Master 127.11.220.254:33109 requested a full tablet report, sending...
I20260925 02:27:26.781041 18309 ts_manager.cc:194] Re-registered known tserver with Master: 210baf669be6468a818f21f2e0f17139 (127.11.220.194:46147)
I20260925 02:27:26.781515 18309 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:44705
I20260925 02:27:26.782830 19426 heartbeater.cc:347] Connected to a master server at 127.11.220.252:44691
I20260925 02:27:26.782883 19426 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:26.782948 19426 heartbeater.cc:511] Master 127.11.220.252:44691 requested a full tablet report, sending...
I20260925 02:27:26.783269 19098 ts_manager.cc:194] Registered new tserver with Master: 210baf669be6468a818f21f2e0f17139 (127.11.220.194:46147)
I20260925 02:27:26.783390 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:39659
--local_ip_for_outbound_sockets=127.11.220.195
--tserver_master_addrs=127.11.220.254:33109
--webserver_port=33709
--webserver_interface=127.11.220.195
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab
--principal=oryx/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.220.254:33109,127.11.220.253:45953
--tserver_master_addrs=127.11.220.254:33109,127.11.220.253:45953,127.11.220.252:44691 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:27:26.783438 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.768732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:38425 (local address 127.11.220.253:45953)
0925 02:27:26.768815 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.768817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.768995 (+ 178us) server_negotiation.cc:407] Connection header received
0925 02:27:26.769969 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.769970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.769981 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.770020 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.770610 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.771261 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.772504 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.772774 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.772833 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.774930 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.774932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.774932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.774938 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.776318 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.776680 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.776681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.776682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.776739 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.782938 (+ 6199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.782940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.782940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.783067 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.783136 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.783278 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.783355 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 02:27:26.783723 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.768740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:45953 (local address 127.11.220.194:38425)
0925 02:27:26.768941 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.768949 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.769836 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.770056 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.770056 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.770398 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.770520 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.770525 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.771326 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.771327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.772075 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.772077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.772839 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.772840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.772892 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.773102 (+ 210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.773106 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.774854 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.776405 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.776406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.776406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.776600 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.776807 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.776808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.776808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.778299 (+ 1491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.783140 (+ 4841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.783140 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.783178 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.783250 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.783344 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":23,"thread_start_us":83,"threads_started":1}
I20260925 02:27:26.784519 19428 heartbeater.cc:347] Connected to a master server at 127.11.220.253:45953
I20260925 02:27:26.784575 19428 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:26.784618 19428 heartbeater.cc:511] Master 127.11.220.253:45953 requested a full tablet report, sending...
I20260925 02:27:26.784906 18913 ts_manager.cc:194] Registered new tserver with Master: 210baf669be6468a818f21f2e0f17139 (127.11.220.194:46147)
W20260925 02:27:26.857933 19433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:26.858106 19433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:26.858129 19433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:26.859184 19433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:26.859212 19433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:26.859236 19433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:26.860553 19433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab
--principal=oryx/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:39659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=33709
--tserver_master_addrs=127.11.220.254:33109,127.11.220.253:45953,127.11.220.252:44691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:26.860711 19433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:26.860859 19433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:26.865129 19439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.865207 19440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:26.865126 19442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:26.865304 19433 server_base.cc:1034] running on GCE node
I20260925 02:27:26.865589 19433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:26.865815 19433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:26.866962 19433 hybrid_clock.cc:648] HybridClock initialized: now 1790303246866950 us; error 21 us; skew 500 ppm
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:26.869326 19433 init.cc:377] Logged in from keytab as oryx/127.11.220.195@KRBTEST.COM (short username oryx)
I20260925 02:27:26.869752 19433 webserver.cc:460] Webserver started at http://127.11.220.195:33709/ using document root <none> and password file <none>
I20260925 02:27:26.869971 19433 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:26.870047 19433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:26.871200 19433 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:26.871662 19449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.871800 19433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.871881 19433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "3dcf8735aa5b4f259eddc6eee227592a"
format_stamp: "Formatted at 2026-09-25 02:27:26 on dist-test-slave-dn4n"
server_key: "59a1a73acb462126e1a201b6ee7e48a0"
server_key_iv: "8a0cef787c66bdbc847dd21013356c02"
server_key_version: "encryptionkey@0"
I20260925 02:27:26.872184 19433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:26.884203 19433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:26.886281 19433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:26.886420 19433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:26.886641 19433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:26.886945 19433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:26.887001 19433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.887033 19433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:26.887060 19433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:26.892575 19433 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:39659
I20260925 02:27:26.892617 19565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:39659 every 8 connection(s)
I20260925 02:27:26.892899 19433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:26.897979 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 19433
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.253@KRBTEST.COM
Sep 25 02:27:26 dist-test-slave-dn4n krb5kdc[18264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303246, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.252@KRBTEST.COM
I20260925 02:27:26.903880 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.893935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:45953 (local address 127.11.220.195:57553)
0925 02:27:26.894035 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.894044 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.894289 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.894597 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.894599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.894830 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.894962 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.894967 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.895708 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.895709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.896761 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.896763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.897171 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.897172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.897253 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.897818 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.897823 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.900717 (+ 2894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.902366 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.902373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.902375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.902647 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.902840 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.902841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.902841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.902961 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.903237 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.903242 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.903389 (+ 147us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.903519 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.903674 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":64,"threads_started":1}
I20260925 02:27:26.904163 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.893761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:44691 (local address 127.11.220.195:47693)
0925 02:27:26.893870 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.893880 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.894278 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.894509 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.894512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.895033 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.895086 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.895088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.895987 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.895988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.896700 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.896701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.897119 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.897120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.897253 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.897767 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.897789 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.901279 (+ 3490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.903142 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.903143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.903144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.903331 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.903542 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.903543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.903543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.903613 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.903872 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.903872 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.903910 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.903972 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.904062 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":25,"thread_start_us":56,"threads_started":1}
I20260925 02:27:26.904292 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.893989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:57553 (local address 127.11.220.253:45953)
0925 02:27:26.894073 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.894075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.894086 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:27:26.894500 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.894502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.894512 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.894564 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.895067 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.895602 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.896856 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.897050 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.897104 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.900788 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.900789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.900790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.900797 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.902273 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.902732 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.902733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.902734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.902791 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.903044 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.903045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.903045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.903157 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.903213 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.904110 (+ 897us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.904182 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 02:27:26.904527 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.893837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:47693 (local address 127.11.220.252:44691)
0925 02:27:26.893885 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.893887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.893975 (+ 88us) server_negotiation.cc:407] Connection header received
0925 02:27:26.894382 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.894384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.894397 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.894426 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.895161 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.895894 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.896778 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.897039 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.897099 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.901336 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.901338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.901338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.901346 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.903065 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.903402 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.903403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.903404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.903474 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.903679 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.903680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.903681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.903792 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.903847 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.904348 (+ 501us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.904422 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 02:27:26.904660 19568 heartbeater.cc:347] Connected to a master server at 127.11.220.253:45953
I20260925 02:27:26.904744 19568 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:26.904831 19566 heartbeater.cc:347] Connected to a master server at 127.11.220.252:44691
I20260925 02:27:26.904860 19566 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:26.904891 19568 heartbeater.cc:511] Master 127.11.220.253:45953 requested a full tablet report, sending...
I20260925 02:27:26.904894 19566 heartbeater.cc:511] Master 127.11.220.252:44691 requested a full tablet report, sending...
I20260925 02:27:26.905279 19098 ts_manager.cc:194] Registered new tserver with Master: 3dcf8735aa5b4f259eddc6eee227592a (127.11.220.195:39659)
I20260925 02:27:26.905531 18913 ts_manager.cc:194] Registered new tserver with Master: 3dcf8735aa5b4f259eddc6eee227592a (127.11.220.195:39659)
I20260925 02:27:26.907083 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.893474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.195:50847)
0925 02:27:26.893637 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.893673 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.894270 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.894511 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.894513 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.894796 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:26.894961 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.894965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.895948 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.895948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.896793 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.896794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.897198 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.897200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.897259 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.897890 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:26.897894 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:26.902020 (+ 4126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:26.904166 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.904167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.904168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.906180 (+ 2012us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.906380 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:26.906380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:26.906381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:26.906429 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:26.906675 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:26.906676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:26.906711 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.906756 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.906836 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":23,"thread_start_us":99,"threads_started":1}
I20260925 02:27:26.907574 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.893516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:50847 (local address 127.11.220.254:33109)
0925 02:27:26.893590 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.893592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.893765 (+ 173us) server_negotiation.cc:407] Connection header received
0925 02:27:26.894382 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.894384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.894396 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.894449 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.895062 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.895865 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.896848 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.897123 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.897183 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.902122 (+ 4939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.902123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.902123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.902131 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.904109 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.906263 (+ 2154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.906265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.906265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.906317 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.906482 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.906483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.906484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.906599 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.906652 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.907358 (+ 706us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.907427 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 02:27:26.907647 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.899058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.11.220.254:33109)
0925 02:27:26.899240 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.899242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.899407 (+ 165us) server_negotiation.cc:407] Connection header received
0925 02:27:26.899430 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.899431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.899443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.899504 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.900310 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.900991 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.901979 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.902296 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.902361 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.903751 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.903752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.903753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.903758 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.906239 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.906696 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.906697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.906697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.906750 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.906924 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.906925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.906925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.907031 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.907116 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.907421 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.907497 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":90,"threads_started":1}
I20260925 02:27:26.907848 19567 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33109
I20260925 02:27:26.907892 19567 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:26.907955 19567 heartbeater.cc:511] Master 127.11.220.254:33109 requested a full tablet report, sending...
I20260925 02:27:26.908164 18309 ts_manager.cc:194] Re-registered known tserver with Master: 3dcf8735aa5b4f259eddc6eee227592a (127.11.220.195:39659)
I20260925 02:27:26.908622 18309 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:50847
I20260925 02:27:26.910161 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.899156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50124 (local address 127.11.220.253:45953)
0925 02:27:26.899415 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.899418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.899542 (+ 124us) server_negotiation.cc:407] Connection header received
0925 02:27:26.899582 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.899583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.899594 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.900313 (+ 719us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.900393 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.901092 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.905160 (+ 4068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.905383 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.905651 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.906671 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.906672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.906672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.906678 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.909043 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.909410 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.909425 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.909425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.909479 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.909701 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.909701 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.909702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.909805 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.909904 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.910010 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.910075 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":46,"threads_started":1}
I20260925 02:27:26.910231 19582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.899354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32828 (local address 127.11.220.252:44691)
0925 02:27:26.899444 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.899446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.899698 (+ 252us) server_negotiation.cc:407] Connection header received
0925 02:27:26.899731 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.899732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.899745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.899792 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:26.900424 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.902741 (+ 2317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.905459 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.905743 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.905796 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.907171 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:26.907172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:26.907173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:26.907178 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:26.908946 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.909471 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.909472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.909473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.909522 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:26.909722 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:26.909723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:26.909723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:26.909830 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:26.909903 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.910089 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.910157 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":45,"threads_started":1}
I20260925 02:27:26.909119 18309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:26.911221 18309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:26.918184 18873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.914610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:39659 (local address 127.0.0.1:37206)
0925 02:27:26.914641 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.914651 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.914686 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.915537 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.915538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.915539 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:26.915563 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.915564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.916315 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.916316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.917300 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.917302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.917970 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.917971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.918027 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.918030 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.918084 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.918127 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260925 02:27:26.918191 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.914683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.11.220.195:39659)
0925 02:27:26.915416 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.915423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.915431 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:26.915459 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.915461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.915473 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.915541 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:26.915624 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.916234 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.917368 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.917891 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.917974 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.918019 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.918097 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.918128 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":95,"threads_started":1}
I20260925 02:27:26.918649 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.914782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:43443 (local address 127.0.0.1:46904)
0925 02:27:26.915230 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.915238 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.915309 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.915545 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.915545 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.915547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:26.915599 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.915601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.916707 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.916707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.917578 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.917579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.918435 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.918436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.918496 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.918500 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.918555 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.918591 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":64,"threads_started":1}
I20260925 02:27:26.918677 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.914880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.11.220.193:43443)
0925 02:27:26.915026 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.915031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.915308 (+ 277us) server_negotiation.cc:407] Connection header received
0925 02:27:26.915380 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.915383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.915400 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.915473 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:26.915669 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.916637 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.917646 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.918345 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.918455 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.918509 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.918568 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.918607 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260925 02:27:26.919015 19497 tablet_service.cc:1467] Processing CreateTablet for tablet a337ec3f753345f88b6bd96c42f3e9e2 (DEFAULT_TABLE table=test-table [id=43439b62e7544d698decd8fc659e18b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:26.919397 19497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a337ec3f753345f88b6bd96c42f3e9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:26.919394 19219 tablet_service.cc:1467] Processing CreateTablet for tablet a337ec3f753345f88b6bd96c42f3e9e2 (DEFAULT_TABLE table=test-table [id=43439b62e7544d698decd8fc659e18b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:26.919653 19219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a337ec3f753345f88b6bd96c42f3e9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:26.921988 19592 tablet_bootstrap.cc:492] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: Bootstrap starting.
I20260925 02:27:26.922660 19592 tablet_bootstrap.cc:654] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:26.922696 19589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.914710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:46147 (local address 127.0.0.1:33080)
0925 02:27:26.914933 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.914964 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.915011 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.915375 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.915376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.915378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:26.915418 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.915419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.916607 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.916608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.918297 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.918300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.922508 (+ 4208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.922509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.922561 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.922564 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.922609 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.922642 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":849,"thread_start_us":70,"threads_started":1}
I20260925 02:27:26.922725 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.915064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33080 (local address 127.11.220.194:46147)
0925 02:27:26.915240 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.915243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.915251 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:26.915281 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.915282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.915294 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.915379 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:26.915471 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.916521 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.918386 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.922435 (+ 4049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.922529 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.922575 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.922625 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.922658 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260925 02:27:26.922967 19592 log.cc:824] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:26.923243 19594 tablet_bootstrap.cc:492] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: Bootstrap starting.
I20260925 02:27:26.923246 19358 tablet_service.cc:1467] Processing CreateTablet for tablet a337ec3f753345f88b6bd96c42f3e9e2 (DEFAULT_TABLE table=test-table [id=43439b62e7544d698decd8fc659e18b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:26.923565 19358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a337ec3f753345f88b6bd96c42f3e9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:26.923666 19592 tablet_bootstrap.cc:492] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: No bootstrap required, opened a new log
I20260925 02:27:26.923740 19592 ts_tablet_manager.cc:1397] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:26.923842 19594 tablet_bootstrap.cc:654] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:26.924131 19594 log.cc:824] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:26.924868 19594 tablet_bootstrap.cc:492] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: No bootstrap required, opened a new log
I20260925 02:27:26.924943 19594 ts_tablet_manager.cc:1397] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:26.925617 19592 raft_consensus.cc:348] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:26.925772 19592 raft_consensus.cc:374] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:26.925808 19592 raft_consensus.cc:729] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dcf8735aa5b4f259eddc6eee227592a, State: Initialized, Role: FOLLOWER
I20260925 02:27:26.925937 19592 consensus_queue.cc:260] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:26.926178 19592 ts_tablet_manager.cc:1428] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:27:26.926839 19598 tablet_bootstrap.cc:492] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: Bootstrap starting.
I20260925 02:27:26.926990 19567 heartbeater.cc:503] Master 127.11.220.254:33109 was elected leader, sending a full tablet report...
I20260925 02:27:26.926963 19594 raft_consensus.cc:348] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:26.927076 19594 raft_consensus.cc:374] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:26.927110 19594 raft_consensus.cc:729] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4236f9f8b9f42de9809d168a1ee5bf6, State: Initialized, Role: FOLLOWER
I20260925 02:27:26.927248 19594 consensus_queue.cc:260] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:26.927470 19594 ts_tablet_manager.cc:1428] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:27:26.927594 19286 heartbeater.cc:503] Master 127.11.220.254:33109 was elected leader, sending a full tablet report...
I20260925 02:27:26.927904 19598 tablet_bootstrap.cc:654] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:26.928227 19598 log.cc:824] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:26.928929 19598 tablet_bootstrap.cc:492] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: No bootstrap required, opened a new log
I20260925 02:27:26.928988 19598 ts_tablet_manager.cc:1397] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:26.930367 19598 raft_consensus.cc:348] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:26.930464 19598 raft_consensus.cc:374] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:26.930482 19598 raft_consensus.cc:729] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210baf669be6468a818f21f2e0f17139, State: Initialized, Role: FOLLOWER
I20260925 02:27:26.930596 19598 consensus_queue.cc:260] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:26.930837 19598 ts_tablet_manager.cc:1428] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:27:26.931118 19427 heartbeater.cc:503] Master 127.11.220.254:33109 was elected leader, sending a full tablet report...
I20260925 02:27:26.971091 19582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.968965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.11.220.252:44691)
0925 02:27:26.969038 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:26.969040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:26.969047 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:26.969070 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:26.969070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:26.969090 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:26.969143 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:26.969251 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.969720 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.970510 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.970857 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.970908 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.970947 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:26.971015 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 02:27:26.971044 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 02:27:26.971092 19589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:26.968883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:44691 (local address 127.0.0.1:32834)
0925 02:27:26.968932 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:26.968948 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:26.968986 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:26.969153 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:26.969154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:26.969156 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:26.969197 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.969198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.969772 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.969773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.970448 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:26.970450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:26.970927 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:26.970928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:26.970967 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:26.970970 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:26.971010 (+ 40us) client_negotiation.cc:240] Negotiation successful
0925 02:27:26.971041 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 02:27:26.971438 19115 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260925 02:27:26.972218 19144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0e02c0ed8aa4340be749d8f7b461dea. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:26.972307 19144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: This master's current role is: LEARNER
I20260925 02:27:27.009479 19599 raft_consensus.cc:482] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:27.009574 19599 raft_consensus.cc:504] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:27.009958 19599 leader_election.cc:288] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3dcf8735aa5b4f259eddc6eee227592a (127.11.220.195:39659), 210baf669be6468a818f21f2e0f17139 (127.11.220.194:46147)
I20260925 02:27:27.012504 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:27.010114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:39659 (local address 127.11.220.193:48389)
0925 02:27:27.010194 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:27.010216 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:27.010265 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:27.010408 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:27.010409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:27.010411 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:27.010484 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:27.010485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.011071 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.011072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:27.011754 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:27.011756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.012243 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.012244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:27.012319 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:27.012322 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:27.012367 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 02:27:27.012396 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":3}
I20260925 02:27:27.012521 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:27.010209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:48389 (local address 127.11.220.195:39659)
0925 02:27:27.010254 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:27.010256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:27.010266 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:27:27.010332 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:27.010333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:27.010342 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:27.010389 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:27.010554 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.010994 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.011840 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.012170 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.012219 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:27.012226 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:27.012392 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 02:27:27.012437 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 02:27:27.012573 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:27.010331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:53397 (local address 127.11.220.194:46147)
0925 02:27:27.010372 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:27.010374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:27.010381 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:27.010405 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:27.010406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:27.010413 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:27.010462 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:27.010619 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.011011 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.011868 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.012219 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.012282 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:27.012289 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:27.012470 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 02:27:27.012504 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:27:27.012621 19289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:27.010231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:46147 (local address 127.11.220.193:53397)
0925 02:27:27.010271 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:27.010280 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:27.010314 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:27.010476 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:27.010477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:27.010478 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:27.010544 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:27.010546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.011110 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.011111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:27.011760 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:27.011762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.012297 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.012298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:27.012373 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:27.012376 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:27.012429 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 02:27:27.012483 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":1}
I20260925 02:27:27.012908 19379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a337ec3f753345f88b6bd96c42f3e9e2" candidate_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210baf669be6468a818f21f2e0f17139" is_pre_election: true
I20260925 02:27:27.013037 19379 raft_consensus.cc:2454] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4236f9f8b9f42de9809d168a1ee5bf6 in term 0.
I20260925 02:27:27.013146 19518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a337ec3f753345f88b6bd96c42f3e9e2" candidate_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dcf8735aa5b4f259eddc6eee227592a" is_pre_election: true
I20260925 02:27:27.013270 19172 leader_election.cc:302] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210baf669be6468a818f21f2e0f17139, c4236f9f8b9f42de9809d168a1ee5bf6; no voters:
I20260925 02:27:27.013288 19518 raft_consensus.cc:2454] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4236f9f8b9f42de9809d168a1ee5bf6 in term 0.
I20260925 02:27:27.013374 19599 raft_consensus.cc:2781] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:27.013422 19599 raft_consensus.cc:482] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:27.013449 19599 raft_consensus.cc:3037] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:27.014240 19599 raft_consensus.cc:504] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:27.014358 19599 leader_election.cc:288] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [CANDIDATE]: Term 1 election: Requested vote from peers 3dcf8735aa5b4f259eddc6eee227592a (127.11.220.195:39659), 210baf669be6468a818f21f2e0f17139 (127.11.220.194:46147)
I20260925 02:27:27.014551 19379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a337ec3f753345f88b6bd96c42f3e9e2" candidate_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210baf669be6468a818f21f2e0f17139"
I20260925 02:27:27.014544 19518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a337ec3f753345f88b6bd96c42f3e9e2" candidate_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dcf8735aa5b4f259eddc6eee227592a"
I20260925 02:27:27.014619 19379 raft_consensus.cc:3037] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:27.014640 19518 raft_consensus.cc:3037] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:27.015249 19379 raft_consensus.cc:2454] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4236f9f8b9f42de9809d168a1ee5bf6 in term 1.
I20260925 02:27:27.015408 19172 leader_election.cc:302] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210baf669be6468a818f21f2e0f17139, c4236f9f8b9f42de9809d168a1ee5bf6; no voters:
I20260925 02:27:27.015497 19599 raft_consensus.cc:2781] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:27.015511 19518 raft_consensus.cc:2454] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4236f9f8b9f42de9809d168a1ee5bf6 in term 1.
I20260925 02:27:27.015681 19599 raft_consensus.cc:686] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 1 LEADER]: Becoming Leader. State: Replica: c4236f9f8b9f42de9809d168a1ee5bf6, State: Running, Role: LEADER
I20260925 02:27:27.015786 19599 consensus_queue.cc:237] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:27.016629 18309 catalog_manager.cc:5495] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4236f9f8b9f42de9809d168a1ee5bf6 (127.11.220.193). New cstate: current_term: 1 leader_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } health_report { overall_health: HEALTHY } } }
I20260925 02:27:27.017155 19076 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: NON_VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: true } } }
I20260925 02:27:27.017264 19076 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:27.017889 19076 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea: attempting to promote NON_VOTER 340bae31edf543e6bebaf5a40119ba4b to VOTER
I20260925 02:27:27.018004 19076 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } }
I20260925 02:27:27.018339 19115 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 LEARNER]: Refusing update from remote peer d0e02c0ed8aa4340be749d8f7b461dea: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260925 02:27:27.018427 18923 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Refusing update from remote peer d0e02c0ed8aa4340be749d8f7b461dea: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260925 02:27:27.018520 19063 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Connected to new peer: Peer: permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 02:27:27.018735 19063 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260925 02:27:27.019076 18875 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 340bae31edf543e6bebaf5a40119ba4b (127.11.220.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
W20260925 02:27:27.019343 19429 tablet.cc:2369] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:27.019243 19115 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 340bae31edf543e6bebaf5a40119ba4b (127.11.220.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:27.019834 18875 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:27.019907 18875 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:27.019960 18875 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:27.020008 18875 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:27.019979 19076 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:27.020047 19076 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:27.020080 19144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:27.020172 19144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:27:27.020339 18923 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 340bae31edf543e6bebaf5a40119ba4b (127.11.220.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:27.021054 19068 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:27.021134 19068 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:27:27.034547 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:27.031502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46918 (local address 127.11.220.193:43443)
0925 02:27:27.031661 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:27.031663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:27.031671 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:27.031701 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:27.031702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:27.031719 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:27.031787 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:27.031953 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.032635 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.033625 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.033949 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.034014 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:27.034223 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:27.034345 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:27.034432 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:27.034489 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 02:27:27.034505 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260925 02:27:27.036008 19518 raft_consensus.cc:1261] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a [term 1 FOLLOWER]: Refusing update from remote peer c4236f9f8b9f42de9809d168a1ee5bf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:27.036023 19379 raft_consensus.cc:1261] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139 [term 1 FOLLOWER]: Refusing update from remote peer c4236f9f8b9f42de9809d168a1ee5bf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:27.036298 19605 consensus_queue.cc:1035] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:27.036389 19599 consensus_queue.cc:1035] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:27.039561 19610 mvcc.cc:205] Tried to move back new op lower bound from 7333082099857199104 to 7333082099777216512. Current Snapshot: MvccSnapshot[applied={T|T < 7333082099857199104}]
I20260925 02:27:27.039952 19609 mvcc.cc:205] Tried to move back new op lower bound from 7333082099857199104 to 7333082099777216512. Current Snapshot: MvccSnapshot[applied={T|T < 7333082099857199104 or (T in {7333082099857199104})}]
I20260925 02:27:27.041448 18308 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:27.041522 18308 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:27.042800 18308 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0e02c0ed8aa4340be749d8f7b461dea: Sending DeleteTablet for 3 replicas of tablet a337ec3f753345f88b6bd96c42f3e9e2
I20260925 02:27:27.043071 19219 tablet_service.cc:1514] Processing DeleteTablet for tablet a337ec3f753345f88b6bd96c42f3e9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:27 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:46904
I20260925 02:27:27.043067 19497 tablet_service.cc:1514] Processing DeleteTablet for tablet a337ec3f753345f88b6bd96c42f3e9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:27 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:37206
I20260925 02:27:27.043349 19358 tablet_service.cc:1514] Processing DeleteTablet for tablet a337ec3f753345f88b6bd96c42f3e9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:27 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:33080
I20260925 02:27:27.043596 19619 tablet_replica.cc:330] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: stopping tablet replica
I20260925 02:27:27.043732 19619 raft_consensus.cc:2229] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:27.043874 19619 raft_consensus.cc:2258] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:27.043955 19617 tablet_replica.cc:330] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: stopping tablet replica
I20260925 02:27:27.044039 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 18277
I20260925 02:27:27.044494 19617 raft_consensus.cc:2229] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:27.044615 19617 raft_consensus.cc:2258] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:27.044720 19619 ts_tablet_manager.cc:1905] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:27.045030 19617 ts_tablet_manager.cc:1905] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:27.046401 19617 ts_tablet_manager.cc:1918] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:27.046468 19617 log.cc:1196] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/a337ec3f753345f88b6bd96c42f3e9e2
I20260925 02:27:27.046610 19619 ts_tablet_manager.cc:1918] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:27.046711 19617 ts_tablet_manager.cc:1939] T a337ec3f753345f88b6bd96c42f3e9e2 P c4236f9f8b9f42de9809d168a1ee5bf6: Deleting consensus metadata
I20260925 02:27:27.046715 19619 log.cc:1196] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/a337ec3f753345f88b6bd96c42f3e9e2
I20260925 02:27:27.047046 19619 ts_tablet_manager.cc:1939] T a337ec3f753345f88b6bd96c42f3e9e2 P 210baf669be6468a818f21f2e0f17139: Deleting consensus metadata
I20260925 02:27:27.047052 19618 tablet_replica.cc:330] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: stopping tablet replica
I20260925 02:27:27.047132 19618 raft_consensus.cc:2229] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:27.047224 19618 raft_consensus.cc:2258] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 02:27:27.050992 19173 connection.cc:535] server connection from 127.0.0.1:46904 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 02:27:27.051235 19312 connection.cc:535] server connection from 127.0.0.1:33080 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 02:27:27.051590 19618 ts_tablet_manager.cc:1905] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:27.053251 19618 ts_tablet_manager.cc:1918] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:27.053328 19618 log.cc:1196] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/a337ec3f753345f88b6bd96c42f3e9e2
I20260925 02:27:27.053579 19618 ts_tablet_manager.cc:1939] T a337ec3f753345f88b6bd96c42f3e9e2 P 3dcf8735aa5b4f259eddc6eee227592a: Deleting consensus metadata
W20260925 02:27:27.053879 19450 connection.cc:441] server connection from 127.0.0.1:37206 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37206 (request call id 1) could send its response
I20260925 02:27:27.060295 19582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:27.043709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32844 (local address 127.11.220.252:44691)
0925 02:27:27.043744 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:27.043746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:27.046398 (+ 2652us) server_negotiation.cc:407] Connection header received
0925 02:27:27.046421 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:27.046422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:27.046433 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:27.046485 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:27.053463 (+ 6978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.054106 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.054939 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.055210 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.055244 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:27.056950 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:27.056951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:27.056952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:27.056959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:27.059095 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:27.059458 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:27.059459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:27.059459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:27.059503 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:27.059703 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:27.059703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:27.059703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:27.059815 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:27.059887 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:27.060116 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 02:27:27.060189 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:27:27.060415 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:27.053061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.11.220.252:44691)
0925 02:27:27.053096 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:27.053097 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:27.053661 (+ 564us) server_negotiation.cc:407] Connection header received
0925 02:27:27.053778 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:27.053779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:27.053788 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:27.053814 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:27.054495 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.055145 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.056084 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.056380 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.056420 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:27.057786 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:27.057788 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:27.057789 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:27.057795 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:27.059344 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:27.059670 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:27.059671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:27.059671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:27.059714 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:27.059887 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:27.059888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:27.059888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:27.059982 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:27.060056 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:27.060276 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 02:27:27.060336 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:27:27.062209 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:27.053222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50126 (local address 127.11.220.253:45953)
0925 02:27:27.053255 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:27.053257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:27.054558 (+ 1301us) server_negotiation.cc:407] Connection header received
0925 02:27:27.054656 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:27.054657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:27.054679 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:27.054707 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:27.055353 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.056040 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.057014 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:27.057301 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:27.057345 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:27.059196 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:27.059197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:27.059197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:27.059203 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:27.061203 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:27.061543 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:27.061544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:27.061544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:27.061591 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:27.061794 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:27.061795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:27.061795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:27.061882 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:27.061979 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:27.062064 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 02:27:27.062149 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:27:28.040234 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.039866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.194:36263)
0925 02:27:28.040104 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.040186 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":57,"threads_started":1}
I20260925 02:27:28.040439 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.040199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.195:35573)
0925 02:27:28.040319 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.040392 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":47,"threads_started":1}
W20260925 02:27:28.040700 19427 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:33109 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
W20260925 02:27:28.040915 19567 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:33109 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
I20260925 02:27:28.041527 19630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.041344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.193:49431)
0925 02:27:28.041444 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.041497 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":52,"threads_started":1}
W20260925 02:27:28.041674 19286 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:33109 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
I20260925 02:27:28.543174 19631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d0e02c0ed8aa4340be749d8f7b461dea)
I20260925 02:27:28.543186 19632 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d0e02c0ed8aa4340be749d8f7b461dea)
I20260925 02:27:28.543314 19631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } }
I20260925 02:27:28.543313 19632 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } }
I20260925 02:27:28.543545 19632 leader_election.cc:288] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109), 340bae31edf543e6bebaf5a40119ba4b (127.11.220.252:44691)
I20260925 02:27:28.543555 19631 leader_election.cc:288] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109), b5a1b01a39994e5da1e75e5f4f4fafb1 (127.11.220.253:45953)
I20260925 02:27:28.543865 19633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.543650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57842)
0925 02:27:28.543778 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.543824 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
W20260925 02:27:28.544014 19089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
I20260925 02:27:28.544507 19633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.544382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57860)
0925 02:27:28.544430 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.544477 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 02:27:28.544513 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.543740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57852)
0925 02:27:28.543908 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.544473 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":43,"threads_started":1}
W20260925 02:27:28.544701 19089 leader_election.cc:334] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109): Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
W20260925 02:27:28.544842 18897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
I20260925 02:27:28.545186 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.545113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57874)
0925 02:27:28.545145 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.545165 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260925 02:27:28.545250 18897 leader_election.cc:334] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109): Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
I20260925 02:27:28.546614 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.543670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:45953 (local address 127.0.0.1:50136)
0925 02:27:28.543854 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.543865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:28.543920 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:28.544140 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:28.544141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:28.544145 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:28.544224 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:28.544227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:28.545089 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:28.545090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:28.545848 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:28.545851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:28.546359 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:28.546360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:28.546420 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:28.546423 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:28.546490 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 02:27:28.546534 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":56,"threads_started":1}
I20260925 02:27:28.546624 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.543738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50136 (local address 127.11.220.253:45953)
0925 02:27:28.543982 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:28.543984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:28.543993 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:28.544018 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:28.544019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:28.544030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:28.544303 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:28.544341 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:28.545014 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:28.545933 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:28.546275 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:28.546359 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:28.546368 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:28.546497 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 02:27:28.546538 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":47,"threads_started":1}
I20260925 02:27:28.546629 19638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.544042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.11.220.252:44691)
0925 02:27:28.544151 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:28.544152 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:28.544158 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:27:28.544174 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:28.544175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:28.544201 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:28.544423 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:28.544470 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:28.545068 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:28.545965 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:28.546296 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:28.546374 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:28.546382 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:28.546511 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 02:27:28.546553 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260925 02:27:28.546629 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.543877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:44691 (local address 127.0.0.1:32870)
0925 02:27:28.544010 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.544018 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:28.544061 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:28.544302 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:28.544302 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:28.544304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:28.544359 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:28.544360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:28.545144 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:28.545145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:28.545879 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:28.545881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:28.546373 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:28.546374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:28.546435 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:28.546438 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:28.546504 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 02:27:28.546546 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":1,"thread_start_us":34,"threads_started":1}
I20260925 02:27:28.546983 18923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "340bae31edf543e6bebaf5a40119ba4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" is_pre_election: true
I20260925 02:27:28.547070 18923 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 340bae31edf543e6bebaf5a40119ba4b in term 1.
I20260925 02:27:28.547061 19115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "340bae31edf543e6bebaf5a40119ba4b" is_pre_election: true
I20260925 02:27:28.547156 19115 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5a1b01a39994e5da1e75e5f4f4fafb1 in term 1.
I20260925 02:27:28.547250 19092 leader_election.cc:302] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 340bae31edf543e6bebaf5a40119ba4b, b5a1b01a39994e5da1e75e5f4f4fafb1; no voters: d0e02c0ed8aa4340be749d8f7b461dea
I20260925 02:27:28.547377 19631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 02:27:28.547422 19631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 FOLLOWER]: Starting leader election (detected failure of leader d0e02c0ed8aa4340be749d8f7b461dea)
I20260925 02:27:28.547420 18897 leader_election.cc:302] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 340bae31edf543e6bebaf5a40119ba4b, b5a1b01a39994e5da1e75e5f4f4fafb1; no voters: d0e02c0ed8aa4340be749d8f7b461dea
I20260925 02:27:28.547451 19631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:27:28.547499 19632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 02:27:28.547529 19632 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d0e02c0ed8aa4340be749d8f7b461dea)
I20260925 02:27:28.547552 19632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:27:28.548341 19631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } }
I20260925 02:27:28.548341 19632 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } }
I20260925 02:27:28.548475 19632 leader_election.cc:288] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [CANDIDATE]: Term 2 election: Requested vote from peers d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109), 340bae31edf543e6bebaf5a40119ba4b (127.11.220.252:44691)
I20260925 02:27:28.548477 19631 leader_election.cc:288] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 2 election: Requested vote from peers d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109), b5a1b01a39994e5da1e75e5f4f4fafb1 (127.11.220.253:45953)
I20260925 02:27:28.548640 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.548557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57886)
0925 02:27:28.548588 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.548616 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260925 02:27:28.548688 18923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "340bae31edf543e6bebaf5a40119ba4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1"
I20260925 02:27:28.548772 18923 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 340bae31edf543e6bebaf5a40119ba4b in current term 2: Already voted for candidate b5a1b01a39994e5da1e75e5f4f4fafb1 in this term.
I20260925 02:27:28.548802 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.548557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57898)
0925 02:27:28.548752 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.548780 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260925 02:27:28.548903 19115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "340bae31edf543e6bebaf5a40119ba4b"
I20260925 02:27:28.548980 19115 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b5a1b01a39994e5da1e75e5f4f4fafb1 in current term 2: Already voted for candidate 340bae31edf543e6bebaf5a40119ba4b in this term.
I20260925 02:27:28.548981 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.548884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57904)
0925 02:27:28.548913 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.548937 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260925 02:27:28.549144 18897 leader_election.cc:334] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109): Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
I20260925 02:27:28.549208 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:28.549098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:57910)
0925 02:27:28.549134 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:28.549175 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260925 02:27:28.549229 18897 leader_election.cc:302] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5a1b01a39994e5da1e75e5f4f4fafb1; no voters: 340bae31edf543e6bebaf5a40119ba4b, d0e02c0ed8aa4340be749d8f7b461dea
W20260925 02:27:28.549268 19089 leader_election.cc:334] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109): Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
I20260925 02:27:28.549290 19632 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 02:27:28.549304 19089 leader_election.cc:302] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 340bae31edf543e6bebaf5a40119ba4b; no voters: b5a1b01a39994e5da1e75e5f4f4fafb1, d0e02c0ed8aa4340be749d8f7b461dea
I20260925 02:27:28.549384 19631 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 02:27:29.041383 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:29.041134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.194:37003)
0925 02:27:29.041285 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:29.041327 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260925 02:27:29.041742 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:29.041235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.195:36737)
0925 02:27:29.041660 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:29.041696 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":66,"threads_started":1}
I20260925 02:27:29.042169 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:29.041961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.193:54657)
0925 02:27:29.042102 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:29.042130 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":51,"threads_started":1}
I20260925 02:27:30.042537 19644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.042100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.194:56433)
0925 02:27:30.042440 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.042482 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":80,"threads_started":1}
I20260925 02:27:30.042594 19645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.042364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.195:34977)
0925 02:27:30.042513 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.042545 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
I20260925 02:27:30.042819 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.042542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.193:59151)
0925 02:27:30.042746 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.042778 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":41,"threads_started":1}
I20260925 02:27:30.095937 19647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:30.096016 19647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } }
I20260925 02:27:30.096208 19647 leader_election.cc:288] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109), b5a1b01a39994e5da1e75e5f4f4fafb1 (127.11.220.253:45953)
I20260925 02:27:30.096418 18923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "340bae31edf543e6bebaf5a40119ba4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" is_pre_election: true
I20260925 02:27:30.096515 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.096316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:58190)
0925 02:27:30.096434 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.096480 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":56,"threads_started":1}
I20260925 02:27:30.096542 18923 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 340bae31edf543e6bebaf5a40119ba4b in term 2.
I20260925 02:27:30.096700 19092 leader_election.cc:302] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 340bae31edf543e6bebaf5a40119ba4b, b5a1b01a39994e5da1e75e5f4f4fafb1; no voters:
I20260925 02:27:30.096773 19647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 02:27:30.096819 19647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:30.096839 19647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 2 FOLLOWER]: Advancing to term 3
I20260925 02:27:30.096870 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.096802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:58200)
0925 02:27:30.096829 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.096851 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260925 02:27:30.096923 19089 leader_election.cc:334] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109): Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
I20260925 02:27:30.097518 19647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } }
I20260925 02:27:30.097659 19647 leader_election.cc:288] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 3 election: Requested vote from peers d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109), b5a1b01a39994e5da1e75e5f4f4fafb1 (127.11.220.253:45953)
I20260925 02:27:30.097815 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.097728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:58210)
0925 02:27:30.097769 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.097792 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 02:27:30.097826 18923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "340bae31edf543e6bebaf5a40119ba4b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1"
I20260925 02:27:30.097913 18923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 2 FOLLOWER]: Advancing to term 3
I20260925 02:27:30.098097 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.098028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:58224)
0925 02:27:30.098054 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.098076 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260925 02:27:30.098186 19089 leader_election.cc:334] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109): Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
I20260925 02:27:30.098641 18923 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 340bae31edf543e6bebaf5a40119ba4b in term 3.
I20260925 02:27:30.098778 19092 leader_election.cc:302] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 340bae31edf543e6bebaf5a40119ba4b, b5a1b01a39994e5da1e75e5f4f4fafb1; no voters: d0e02c0ed8aa4340be749d8f7b461dea
I20260925 02:27:30.098866 19647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 3 FOLLOWER]: Leader election won for term 3
I20260925 02:27:30.098961 19647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [term 3 LEADER]: Becoming Leader. State: Replica: 340bae31edf543e6bebaf5a40119ba4b, State: Running, Role: LEADER
I20260925 02:27:30.099035 19647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } }
I20260925 02:27:30.099375 19650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 340bae31edf543e6bebaf5a40119ba4b. Latest consensus state: current_term: 3 leader_uuid: "340bae31edf543e6bebaf5a40119ba4b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:30.099465 19650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:30.099630 19653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:30.099985 19653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:30.100040 19653 catalog_manager.cc:1259] Loaded cluster ID: 013950402d7b4ece8b5dc4d4425796f1
I20260925 02:27:30.100071 19653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:30.100205 19653 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:30.100252 19653 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Loaded TSK: 0
I20260925 02:27:30.100342 19653 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:27:30.136991 19098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:30.138622 19098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:30.139516 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.139412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:58230)
0925 02:27:30.139464 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.139494 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 02:27:30.139539 18923 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [term 3 FOLLOWER]: Refusing update from remote peer 340bae31edf543e6bebaf5a40119ba4b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260925 02:27:30.139813 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.139732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.0.0.1:58246)
0925 02:27:30.139771 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.139796 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 02:27:30.139765 19647 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260925 02:27:30.139899 19089 consensus_peers.cc:487] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b -> Peer d0e02c0ed8aa4340be749d8f7b461dea (127.11.220.254:33109): Couldn't send request to peer d0e02c0ed8aa4340be749d8f7b461dea. Status: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 02:27:30.140345 19655 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 340bae31edf543e6bebaf5a40119ba4b. Latest consensus state: current_term: 3 leader_uuid: "340bae31edf543e6bebaf5a40119ba4b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:30.140465 19655 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:27:30.140895 19647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "340bae31edf543e6bebaf5a40119ba4b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:30.140995 19647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:30.141203 19655 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "340bae31edf543e6bebaf5a40119ba4b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d0e02c0ed8aa4340be749d8f7b461dea" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33109 } } peers { permanent_uuid: "b5a1b01a39994e5da1e75e5f4f4fafb1" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 45953 } attrs { promote: false } } peers { permanent_uuid: "340bae31edf543e6bebaf5a40119ba4b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 44691 } attrs { promote: false } } }
I20260925 02:27:30.141288 19655 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a1b01a39994e5da1e75e5f4f4fafb1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:27:30.141503 19656 mvcc.cc:205] Tried to move back new op lower bound from 7333082112569798656 to 7333082112406212608. Current Snapshot: MvccSnapshot[applied={T|T < 7333082112569798656}]
I20260925 02:27:30.145714 19661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.142595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:39659 (local address 127.0.0.1:37214)
0925 02:27:30.142759 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.142767 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:30.142807 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:30.143190 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:30.143191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:30.143193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:30.143240 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:30.143242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.144064 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.144065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:30.144893 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:30.144896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.145487 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.145488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:30.145550 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.145555 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:27:30.145606 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:27:30.145648 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":83,"threads_started":1}
I20260925 02:27:30.145735 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.142769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37214 (local address 127.11.220.195:39659)
0925 02:27:30.142879 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:30.142881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:30.142889 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:30.142917 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:30.142918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:30.142928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:30.142995 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:30.143333 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.144010 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.144981 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.145418 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.145474 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.145481 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:30.145629 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 02:27:30.145668 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":58,"threads_started":1}
I20260925 02:27:30.145792 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.142572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:43443 (local address 127.0.0.1:46932)
0925 02:27:30.142712 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.142722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:30.142760 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:30.143044 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:30.143045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:30.143047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:30.143096 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:30.143098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.143775 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.143776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:30.144889 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:30.144892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.145563 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.145564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:30.145617 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.145619 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:30.145644 (+ 25us) client_negotiation.cc:240] Negotiation successful
0925 02:27:30.145682 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260925 02:27:30.146169 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.142749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46932 (local address 127.11.220.193:43443)
0925 02:27:30.142881 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:30.142884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:30.142895 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:27:30.142923 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:30.142924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:30.142936 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:30.143004 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:30.143183 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.143688 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.144980 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.145481 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.145873 (+ 392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.145888 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:30.145928 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 02:27:30.146092 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":48,"threads_started":1}
I20260925 02:27:30.146178 19497 tablet_service.cc:1467] Processing CreateTablet for tablet 18b768a4f8b14138ad0ac82dad4c5c87 (DEFAULT_TABLE table=test-table [id=4e1fad887e974ab8a0299b8cbde4ae28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:30.146302 19497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b768a4f8b14138ad0ac82dad4c5c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:30.146368 19219 tablet_service.cc:1467] Processing CreateTablet for tablet 18b768a4f8b14138ad0ac82dad4c5c87 (DEFAULT_TABLE table=test-table [id=4e1fad887e974ab8a0299b8cbde4ae28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:30.146474 19219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b768a4f8b14138ad0ac82dad4c5c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:30.146653 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.142512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:46147 (local address 127.0.0.1:33096)
0925 02:27:30.142544 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.142554 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:30.142611 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:30.143106 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:30.143106 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:30.143108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:30.143138 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:30.143139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.144016 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.144017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:30.145033 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:30.145034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.146416 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.146417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:30.146483 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.146488 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:27:30.146551 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 02:27:30.146599 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 02:27:30.146667 19660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.142575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.11.220.194:46147)
0925 02:27:30.142923 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:30.142925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:30.142993 (+ 68us) server_negotiation.cc:407] Connection header received
0925 02:27:30.143018 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:30.143019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:30.143028 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:30.143095 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:30.143207 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.143940 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.145508 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.146344 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.146419 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.146428 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:30.146565 (+ 137us) server_negotiation.cc:299] Negotiation successful
0925 02:27:30.146603 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":69,"threads_started":1}
I20260925 02:27:30.147617 19666 tablet_bootstrap.cc:492] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a: Bootstrap starting.
I20260925 02:27:30.147679 19667 tablet_bootstrap.cc:492] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6: Bootstrap starting.
I20260925 02:27:30.147943 19358 tablet_service.cc:1467] Processing CreateTablet for tablet 18b768a4f8b14138ad0ac82dad4c5c87 (DEFAULT_TABLE table=test-table [id=4e1fad887e974ab8a0299b8cbde4ae28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:30.148037 19667 tablet_bootstrap.cc:654] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:30.148057 19358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b768a4f8b14138ad0ac82dad4c5c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:30.148618 19667 tablet_bootstrap.cc:492] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6: No bootstrap required, opened a new log
I20260925 02:27:30.148665 19667 ts_tablet_manager.cc:1397] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:30.148770 19667 raft_consensus.cc:348] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:30.148831 19667 raft_consensus.cc:374] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:30.148851 19667 raft_consensus.cc:729] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4236f9f8b9f42de9809d168a1ee5bf6, State: Initialized, Role: FOLLOWER
I20260925 02:27:30.148895 19667 consensus_queue.cc:260] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:30.149048 19667 ts_tablet_manager.cc:1428] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:30.149348 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.149257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.193:51973)
0925 02:27:30.149292 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.149322 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 02:27:30.149626 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.149533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.193:34889)
0925 02:27:30.149578 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.149602 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 02:27:30.149694 19666 tablet_bootstrap.cc:654] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:30.149704 19285 heartbeater.cc:503] Master 127.11.220.252:44691 was elected leader, sending a full tablet report...
I20260925 02:27:30.149978 19670 tablet_bootstrap.cc:492] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139: Bootstrap starting.
I20260925 02:27:30.150250 19666 tablet_bootstrap.cc:492] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a: No bootstrap required, opened a new log
I20260925 02:27:30.150290 19666 ts_tablet_manager.cc:1397] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 02:27:30.150368 19666 raft_consensus.cc:348] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:30.150424 19666 raft_consensus.cc:374] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:30.150435 19666 raft_consensus.cc:729] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dcf8735aa5b4f259eddc6eee227592a, State: Initialized, Role: FOLLOWER
I20260925 02:27:30.150483 19666 consensus_queue.cc:260] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:30.150643 19666 ts_tablet_manager.cc:1428] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:30.150919 19645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.150833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.195:40853)
0925 02:27:30.150867 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.150895 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 02:27:30.150977 19670 tablet_bootstrap.cc:654] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:30.151131 19566 heartbeater.cc:503] Master 127.11.220.252:44691 was elected leader, sending a full tablet report...
I20260925 02:27:30.151396 19645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.151318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.195:38121)
0925 02:27:30.151349 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.151371 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260925 02:27:30.151770 19670 tablet_bootstrap.cc:492] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139: No bootstrap required, opened a new log
I20260925 02:27:30.151805 19670 ts_tablet_manager.cc:1397] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:30.151912 19670 raft_consensus.cc:348] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:30.152002 19670 raft_consensus.cc:374] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:30.152019 19670 raft_consensus.cc:729] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210baf669be6468a818f21f2e0f17139, State: Initialized, Role: FOLLOWER
I20260925 02:27:30.152055 19670 consensus_queue.cc:260] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:30.152367 19670 ts_tablet_manager.cc:1428] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:30.152525 19644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.152424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.194:46957)
0925 02:27:30.152478 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.152505 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 02:27:30.152776 19426 heartbeater.cc:503] Master 127.11.220.252:44691 was elected leader, sending a full tablet report...
I20260925 02:27:30.152787 19644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.152704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.194:46285)
0925 02:27:30.152736 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.152761 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260925 02:27:30.153738 19288 tablet.cc:2369] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:30.295205 19672 raft_consensus.cc:482] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:30.295286 19672 raft_consensus.cc:504] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:30.295689 19672 leader_election.cc:288] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 210baf669be6468a818f21f2e0f17139 (127.11.220.194:46147), c4236f9f8b9f42de9809d168a1ee5bf6 (127.11.220.193:43443)
I20260925 02:27:30.299360 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.295969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:52251 (local address 127.11.220.193:43443)
0925 02:27:30.296012 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:30.296014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:30.296184 (+ 170us) server_negotiation.cc:407] Connection header received
0925 02:27:30.296215 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:30.296216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:30.296223 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:30.296261 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:30.296405 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.297045 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.298475 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.299020 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.299091 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.299100 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:30.299258 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 02:27:30.299294 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 02:27:30.299360 19660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.295938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:52271 (local address 127.11.220.194:46147)
0925 02:27:30.295993 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:30.295995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:30.296025 (+ 30us) server_negotiation.cc:407] Connection header received
0925 02:27:30.296052 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:30.296053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:30.296063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:30.296111 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:30.296309 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.297028 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.298475 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.299020 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.299091 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.299100 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:30.299258 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 02:27:30.299294 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:27:30.299364 19645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.295867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:46147 (local address 127.11.220.195:52271)
0925 02:27:30.295924 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.295936 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:30.295989 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:30.296136 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:30.296137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:30.296139 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:30.296214 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:30.296216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.297112 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.297113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:30.298388 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:30.298391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.299099 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.299100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:30.299168 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.299173 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:27:30.299229 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 02:27:30.299276 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":54}
I20260925 02:27:30.299373 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.295869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:43443 (local address 127.11.220.195:52251)
0925 02:27:30.296081 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.296087 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:30.296160 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:30.296285 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:30.296285 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:30.296287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:30.296331 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:30.296332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.297119 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.297119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:30.298388 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:30.298391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.299139 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.299140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:30.299191 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.299194 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:30.299241 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 02:27:30.299276 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":88,"thread_start_us":106,"threads_started":1}
I20260925 02:27:30.299645 19379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b768a4f8b14138ad0ac82dad4c5c87" candidate_uuid: "3dcf8735aa5b4f259eddc6eee227592a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210baf669be6468a818f21f2e0f17139" is_pre_election: true
I20260925 02:27:30.299719 19379 raft_consensus.cc:2454] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dcf8735aa5b4f259eddc6eee227592a in term 0.
I20260925 02:27:30.299685 19239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b768a4f8b14138ad0ac82dad4c5c87" candidate_uuid: "3dcf8735aa5b4f259eddc6eee227592a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" is_pre_election: true
I20260925 02:27:30.299835 19239 raft_consensus.cc:2454] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dcf8735aa5b4f259eddc6eee227592a in term 0.
I20260925 02:27:30.299878 19451 leader_election.cc:302] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210baf669be6468a818f21f2e0f17139, 3dcf8735aa5b4f259eddc6eee227592a; no voters:
I20260925 02:27:30.300041 19672 raft_consensus.cc:2781] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:30.300112 19672 raft_consensus.cc:482] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:30.300143 19672 raft_consensus.cc:3037] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:30.300876 19672 raft_consensus.cc:504] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:30.301004 19672 leader_election.cc:288] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [CANDIDATE]: Term 1 election: Requested vote from peers 210baf669be6468a818f21f2e0f17139 (127.11.220.194:46147), c4236f9f8b9f42de9809d168a1ee5bf6 (127.11.220.193:43443)
I20260925 02:27:30.301157 19379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b768a4f8b14138ad0ac82dad4c5c87" candidate_uuid: "3dcf8735aa5b4f259eddc6eee227592a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210baf669be6468a818f21f2e0f17139"
I20260925 02:27:30.301170 19239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b768a4f8b14138ad0ac82dad4c5c87" candidate_uuid: "3dcf8735aa5b4f259eddc6eee227592a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6"
I20260925 02:27:30.301237 19239 raft_consensus.cc:3037] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:30.301241 19379 raft_consensus.cc:3037] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:30.301756 19379 raft_consensus.cc:2454] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dcf8735aa5b4f259eddc6eee227592a in term 1.
I20260925 02:27:30.301934 19239 raft_consensus.cc:2454] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dcf8735aa5b4f259eddc6eee227592a in term 1.
I20260925 02:27:30.301949 19451 leader_election.cc:302] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210baf669be6468a818f21f2e0f17139, 3dcf8735aa5b4f259eddc6eee227592a; no voters:
I20260925 02:27:30.302062 19672 raft_consensus.cc:2781] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:30.302222 19672 raft_consensus.cc:686] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 1 LEADER]: Becoming Leader. State: Replica: 3dcf8735aa5b4f259eddc6eee227592a, State: Running, Role: LEADER
I20260925 02:27:30.302299 19672 consensus_queue.cc:237] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } } peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } }
I20260925 02:27:30.302529 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.302429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.195:58919)
0925 02:27:30.302477 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.302508 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 02:27:30.303217 19098 catalog_manager.cc:5495] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3dcf8735aa5b4f259eddc6eee227592a (127.11.220.195). New cstate: current_term: 1 leader_uuid: "3dcf8735aa5b4f259eddc6eee227592a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3dcf8735aa5b4f259eddc6eee227592a" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 } health_report { overall_health: UNKNOWN } } }
I20260925 02:27:30.325444 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.323481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37224 (local address 127.11.220.195:39659)
0925 02:27:30.323533 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:30.323534 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:30.323542 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:30.323567 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:30.323568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:30.323582 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:30.323659 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:30.323768 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.324162 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.324810 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:30.324985 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:30.325042 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:30.325193 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:30.325288 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:30.325336 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:30.325363 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 02:27:30.325378 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 02:27:30.326454 19379 raft_consensus.cc:1261] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139 [term 1 FOLLOWER]: Refusing update from remote peer 3dcf8735aa5b4f259eddc6eee227592a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:30.326548 19239 raft_consensus.cc:1261] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6 [term 1 FOLLOWER]: Refusing update from remote peer 3dcf8735aa5b4f259eddc6eee227592a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:30.326685 19672 consensus_queue.cc:1035] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [LEADER]: Connected to new peer: Peer: permanent_uuid: "210baf669be6468a818f21f2e0f17139" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:30.326797 19676 consensus_queue.cc:1035] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4236f9f8b9f42de9809d168a1ee5bf6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 43443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:30.327824 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.327646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.193:45087)
0925 02:27:30.327769 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.327796 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260925 02:27:30.327829 19644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.327656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.194:40213)
0925 02:27:30.327765 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.327794 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 02:27:30.328644 19644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.328518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.194:55651)
0925 02:27:30.328590 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.328620 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 02:27:30.330690 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.330592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.193:58301)
0925 02:27:30.330634 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.330662 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 02:27:30.330698 19098 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:30.330768 19098 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:30.330945 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.328572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.195:55025)
0925 02:27:30.330886 (+ 2314us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.330921 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2299}
I20260925 02:27:30.331044 19683 mvcc.cc:205] Tried to move back new op lower bound from 7333082113335271424 to 7333082113239470080. Current Snapshot: MvccSnapshot[applied={T|T < 7333082113335271424 or (T in {7333082113335271424})}]
I20260925 02:27:30.331233 19645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:30.331158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33109 (local address 127.11.220.195:46435)
0925 02:27:30.331189 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.331209 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:33109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 02:27:30.332077 19098 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 340bae31edf543e6bebaf5a40119ba4b: Sending DeleteTablet for 3 replicas of tablet 18b768a4f8b14138ad0ac82dad4c5c87
I20260925 02:27:30.332309 19219 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b768a4f8b14138ad0ac82dad4c5c87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:30 UTC) from {username='oryx', principal='oryx/127.11.220.252@KRBTEST.COM'} at 127.0.0.1:46932
I20260925 02:27:30.332332 19358 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b768a4f8b14138ad0ac82dad4c5c87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:30 UTC) from {username='oryx', principal='oryx/127.11.220.252@KRBTEST.COM'} at 127.0.0.1:33096
I20260925 02:27:30.332517 19497 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b768a4f8b14138ad0ac82dad4c5c87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:30 UTC) from {username='oryx', principal='oryx/127.11.220.252@KRBTEST.COM'} at 127.0.0.1:37214
I20260925 02:27:30.332728 19692 tablet_replica.cc:330] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a: stopping tablet replica
I20260925 02:27:30.332739 19690 tablet_replica.cc:330] T 18b768a4f8b14138ad0ac82dad4c5c87 P c4236f9f8b9f42de9809d168a1ee5bf6: stopping tablet replica
I20260925 02:27:30.332752 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 19064
I20260925 02:27:30.333534 19692 raft_consensus.cc:2229] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:30.333657 19692 raft_consensus.cc:2258] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:30.334013 19692 ts_tablet_manager.cc:1905] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:30.335292 19691 tablet_replica.cc:330] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139: stopping tablet replica
I20260925 02:27:30.335391 19691 raft_consensus.cc:2229] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:30.335475 19691 raft_consensus.cc:2258] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:30.335726 19692 ts_tablet_manager.cc:1918] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:30.335769 19691 ts_tablet_manager.cc:1905] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:30.335844 19692 log.cc:1196] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/18b768a4f8b14138ad0ac82dad4c5c87
I20260925 02:27:30.336205 19692 ts_tablet_manager.cc:1939] T 18b768a4f8b14138ad0ac82dad4c5c87 P 3dcf8735aa5b4f259eddc6eee227592a: Deleting consensus metadata
I20260925 02:27:30.336553 19090 catalog_manager.cc:4841] TS 3dcf8735aa5b4f259eddc6eee227592a (127.11.220.195:39659): tablet 18b768a4f8b14138ad0ac82dad4c5c87 (table test-table [id=4e1fad887e974ab8a0299b8cbde4ae28]) successfully deleted
I20260925 02:27:30.337576 19691 ts_tablet_manager.cc:1918] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:30.337630 19691 log.cc:1196] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/18b768a4f8b14138ad0ac82dad4c5c87
I20260925 02:27:30.337823 19691 ts_tablet_manager.cc:1939] T 18b768a4f8b14138ad0ac82dad4c5c87 P 210baf669be6468a818f21f2e0f17139: Deleting consensus metadata
I20260925 02:27:30.338141 19090 catalog_manager.cc:4841] TS 210baf669be6468a818f21f2e0f17139 (127.11.220.194:46147): tablet 18b768a4f8b14138ad0ac82dad4c5c87 (table test-table [id=4e1fad887e974ab8a0299b8cbde4ae28]) successfully deleted
W20260925 02:27:30.340693 19648 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:30.340559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:43443 (local address 127.0.0.1:46934)
0925 02:27:30.340631 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:30.340651 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:43443: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":56}
W20260925 02:27:30.340791 19092 catalog_manager.cc:4584] TS c4236f9f8b9f42de9809d168a1ee5bf6 (127.11.220.193:43443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 18b768a4f8b14138ad0ac82dad4c5c87: Network error: Client connection negotiation failed: client connection to 127.11.220.193:43443: connect: Connection reset by peer (error 104)
I20260925 02:27:30.340930 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 19293
I20260925 02:27:30.346652 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 19433
I20260925 02:27:30.353413 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 18778
I20260925 02:27:30.358361 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 18969
2026-09-25T02:27:30Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5928 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:30 dist-test-slave-dn4n krb5kdc[19695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:30 dist-test-slave-dn4n krb5kdc[19695](info): set up 2 sockets
Sep 25 02:27:30 dist-test-slave-dn4n krb5kdc[19695](info): commencing operation
krb5kdc: starting...
W20260925 02:27:31.297462 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:31 dist-test-slave-dn4n krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.254.keytab.
Entry for principal oryx/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.254.keytab.
I20260925 02:27:31.420817 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43463
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.254.keytab
--principal=oryx/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:43463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:31.491820 19708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:31.492017 19708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:31.492051 19708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:31.493104 19708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:31.493170 19708 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 02:27:31.493196 19708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:31.493224 19708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:31.493247 19708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:31.493271 19708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:31.494673 19708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:43463
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.254.keytab
--principal=oryx/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:31.494925 19708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:31.495132 19708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:31.499264 19714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:31.499316 19716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:31.499292 19713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:31.499339 19708 server_base.cc:1034] running on GCE node
I20260925 02:27:31.499541 19708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:31.499802 19708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:31.500952 19708 hybrid_clock.cc:648] HybridClock initialized: now 1790303251500933 us; error 33 us; skew 500 ppm
Sep 25 02:27:31 dist-test-slave-dn4n krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:31.502985 19708 init.cc:377] Logged in from keytab as oryx/127.11.220.254@KRBTEST.COM (short username oryx)
I20260925 02:27:31.503346 19708 webserver.cc:460] Webserver started at http://127.11.220.254:39945/ using document root <none> and password file <none>
I20260925 02:27:31.503546 19708 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:31.503607 19708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:31.503746 19708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:31.504730 19708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "93575dcbd5f0463ab2973d0eb36f5eb9"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "19d3ea12c8dd3d5c42c66f6ab8c1968e"
server_key_iv: "e82b1c9c32365fd7ddee301d73358217"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.505043 19708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "93575dcbd5f0463ab2973d0eb36f5eb9"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "19d3ea12c8dd3d5c42c66f6ab8c1968e"
server_key_iv: "e82b1c9c32365fd7ddee301d73358217"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.506170 19708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:31.506829 19724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:31.506991 19708 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:31.507082 19708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "93575dcbd5f0463ab2973d0eb36f5eb9"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "19d3ea12c8dd3d5c42c66f6ab8c1968e"
server_key_iv: "e82b1c9c32365fd7ddee301d73358217"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.507165 19708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:31.521567 19708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:31.522910 19708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:31.523039 19708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:31.527096 19708 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:43463
I20260925 02:27:31.527159 19787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:43463 every 8 connection(s)
I20260925 02:27:31.527432 19708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:31.528059 19788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:31.529896 19788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9: Bootstrap starting.
I20260925 02:27:31.530366 19788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:31.530586 19788 log.cc:824] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:31.531088 19788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9: No bootstrap required, opened a new log
I20260925 02:27:31.532827 19788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93575dcbd5f0463ab2973d0eb36f5eb9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43463 } }
I20260925 02:27:31.532966 19788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:31.532986 19788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93575dcbd5f0463ab2973d0eb36f5eb9, State: Initialized, Role: FOLLOWER
I20260925 02:27:31.533097 19788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93575dcbd5f0463ab2973d0eb36f5eb9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43463 } }
I20260925 02:27:31.533172 19788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:31.533193 19788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:31.533226 19788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:31.533653 19788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93575dcbd5f0463ab2973d0eb36f5eb9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43463 } }
I20260925 02:27:31.533727 19788 leader_election.cc:302] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93575dcbd5f0463ab2973d0eb36f5eb9; no voters:
I20260925 02:27:31.533852 19788 leader_election.cc:288] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:31.533919 19791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:31.534062 19791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [term 1 LEADER]: Becoming Leader. State: Replica: 93575dcbd5f0463ab2973d0eb36f5eb9, State: Running, Role: LEADER
I20260925 02:27:31.534092 19788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:31.534142 19791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93575dcbd5f0463ab2973d0eb36f5eb9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43463 } }
I20260925 02:27:31.534507 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 19708
I20260925 02:27:31.534602 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:31.534838 12147 external_mini_cluster.cc:1428] Setting key 33f9c038e2f7177668ec454092ebbca4
I20260925 02:27:31.535279 19792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93575dcbd5f0463ab2973d0eb36f5eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93575dcbd5f0463ab2973d0eb36f5eb9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43463 } } }
I20260925 02:27:31.535360 19792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:31.535454 19793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93575dcbd5f0463ab2973d0eb36f5eb9. Latest consensus state: current_term: 1 leader_uuid: "93575dcbd5f0463ab2973d0eb36f5eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93575dcbd5f0463ab2973d0eb36f5eb9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43463 } } }
I20260925 02:27:31.535691 19793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9 [sys.catalog]: This master's current role is: LEADER
W20260925 02:27:31.535833 19808 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:27:31.535876 19808 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:27:31.535938 19811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:31.536130 19811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:31.537925 19811 catalog_manager.cc:1347] Generated new cluster ID: 03b3df66b01f42978a5eb48d4d402f43
I20260925 02:27:31.537973 19811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:27:31 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:31.545267 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.535411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.11.220.254:43463)
0925 02:27:31.535561 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:31.535569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:31.535599 (+ 30us) server_negotiation.cc:407] Connection header received
0925 02:27:31.535799 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:31.535806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:31.535853 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:31.535969 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:31.536760 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.537797 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.538776 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.539413 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.539529 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.542033 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:31.542037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:31.542038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:31.542053 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:31.543791 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.544158 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.544160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.544160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.544247 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.544441 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.544442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.544442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.544713 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:31.544833 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:31.544954 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 02:27:31.545061 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260925 02:27:31.552469 19811 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:31.553119 19811 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:31.564465 19811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9: Generated new TSK 0
I20260925 02:27:31.564635 19811 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab.
Entry for principal oryx/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab.
I20260925 02:27:31.593261 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43463
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab
--principal=oryx/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:31.666558 19815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:31.666749 19815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:31.666785 19815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:31.667825 19815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:31.667882 19815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:31.667912 19815 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:27:31.667946 19815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:31.669482 19815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab
--principal=oryx/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.220.254:43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:31.669735 19815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:31.669960 19815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:31.674463 19821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:31.674476 19823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:31.674609 19815 server_base.cc:1034] running on GCE node
W20260925 02:27:31.674476 19820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:31.674850 19815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:31.675067 19815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:31.676200 19815 hybrid_clock.cc:648] HybridClock initialized: now 1790303251676190 us; error 25 us; skew 500 ppm
Sep 25 02:27:31 dist-test-slave-dn4n krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:31.678546 19815 init.cc:377] Logged in from keytab as oryx/127.11.220.193@KRBTEST.COM (short username oryx)
I20260925 02:27:31.678936 19815 webserver.cc:460] Webserver started at http://127.11.220.193:39885/ using document root <none> and password file <none>
I20260925 02:27:31.679160 19815 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:31.679239 19815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:31.679379 19815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:31.680357 19815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "80c3f96980d44346847183a86a96cc25"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "8da2b7d365ccc9a302f782570ed58b4d"
server_key_iv: "8af3dda396fe9913a236e7a8ab004fff"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.680723 19815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "80c3f96980d44346847183a86a96cc25"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "8da2b7d365ccc9a302f782570ed58b4d"
server_key_iv: "8af3dda396fe9913a236e7a8ab004fff"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.681911 19815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:31.682557 19830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:31.682730 19815 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:31.682817 19815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "80c3f96980d44346847183a86a96cc25"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "8da2b7d365ccc9a302f782570ed58b4d"
server_key_iv: "8af3dda396fe9913a236e7a8ab004fff"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.682894 19815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:31.702287 19815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:31.703917 19815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:31.704083 19815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:31.704581 19815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:31.704636 19815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:31.704665 19815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:31.704695 19815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:27:31 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:31.711618 19815 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:41993
I20260925 02:27:31.711669 19948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:41993 every 8 connection(s)
I20260925 02:27:31.711920 19815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:31.714389 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.704982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:34687 (local address 127.11.220.254:43463)
0925 02:27:31.705049 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:31.705051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:31.705280 (+ 229us) server_negotiation.cc:407] Connection header received
0925 02:27:31.705924 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:31.705925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:31.705937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:31.705992 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:31.706823 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.707585 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.709240 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.709549 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.709635 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.711800 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:31.711802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:31.711802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:31.711811 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:31.713120 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.713471 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.713472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.713473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.713527 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.713769 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.713770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.713771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.713878 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:31.713951 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:31.714213 (+ 262us) server_negotiation.cc:299] Negotiation successful
0925 02:27:31.714304 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:27:31.714526 19839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.704968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43463 (local address 127.11.220.193:34687)
0925 02:27:31.705140 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:31.705186 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:31.705820 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:31.706074 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:31.706078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:31.706433 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:31.706710 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.706716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.707688 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.707690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.709154 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.709157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.709653 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.709654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.709785 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.710230 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:31.710245 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:31.711724 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:31.713198 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.713205 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.713207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.713400 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.713595 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.713596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.713597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.713702 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.713939 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:31.713946 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:31.714086 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 02:27:31.714188 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 02:27:31.714325 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260925 02:27:31.715003 19949 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43463
I20260925 02:27:31.715087 19949 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:31.715255 19949 heartbeater.cc:511] Master 127.11.220.254:43463 requested a full tablet report, sending...
I20260925 02:27:31.715775 19741 ts_manager.cc:194] Registered new tserver with Master: 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993)
I20260925 02:27:31.716367 19741 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:34687
I20260925 02:27:31.717283 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 19815
I20260925 02:27:31.717358 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:31.717551 12147 external_mini_cluster.cc:1428] Setting key a7889df94fe6e38928dda87d24ffa167
I20260925 02:27:31.721462 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.715936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:46791 (local address 127.11.220.254:43463)
0925 02:27:31.715972 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:31.715974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:31.716089 (+ 115us) server_negotiation.cc:407] Connection header received
0925 02:27:31.716154 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:31.716154 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:31.716163 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:31.716211 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:31.716529 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.717051 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.717860 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.718108 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.718174 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.719072 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:31.719073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:31.719073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:31.719079 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:31.720394 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.720755 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.720757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.720757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.720811 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.720999 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.721000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.721001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.721098 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:31.721177 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:31.721323 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 02:27:31.721385 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:27:31.721477 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.715881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43463 (local address 127.11.220.193:46791)
0925 02:27:31.716031 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:31.716039 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:31.716101 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:31.716216 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:31.716217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:31.716378 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:31.716440 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.716442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.717147 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.717148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.717804 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.717806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.718189 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.718190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.718266 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.718609 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:31.718614 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:31.718996 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:31.720509 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.720510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.720511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.720676 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.720876 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.720877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.720877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.720933 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.721169 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:31.721170 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:31.721223 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 02:27:31.721298 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:27:31.721386 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for oryx/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab.
Entry for principal oryx/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab.
I20260925 02:27:31.739267 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43463
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab
--principal=oryx/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:31.810195 19958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:31.810340 19958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:31.810359 19958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:31.811365 19958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:31.811399 19958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:31.811417 19958 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:27:31.811468 19958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:31.812778 19958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab
--principal=oryx/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.220.254:43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:31.812980 19958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:31.813175 19958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:31.817651 19964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:31.817634 19966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:31.817655 19963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:31.817775 19958 server_base.cc:1034] running on GCE node
I20260925 02:27:31.818043 19958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:31.818288 19958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:31.819471 19958 hybrid_clock.cc:648] HybridClock initialized: now 1790303251819460 us; error 46 us; skew 500 ppm
Sep 25 02:27:31 dist-test-slave-dn4n krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:31.821775 19958 init.cc:377] Logged in from keytab as oryx/127.11.220.194@KRBTEST.COM (short username oryx)
I20260925 02:27:31.822182 19958 webserver.cc:460] Webserver started at http://127.11.220.194:37143/ using document root <none> and password file <none>
I20260925 02:27:31.822374 19958 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:31.822431 19958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:31.822548 19958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:31.823414 19958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "92bad131b5a04a85bcab8632eac07cc5"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "287f7ff08dc8662e752d59b5ab486ac3"
server_key_iv: "cacae1091e9ac4be1979f3286cf7ea4d"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.823756 19958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "92bad131b5a04a85bcab8632eac07cc5"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "287f7ff08dc8662e752d59b5ab486ac3"
server_key_iv: "cacae1091e9ac4be1979f3286cf7ea4d"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.825044 19958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:31.825670 19973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:31.825819 19958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:31.825904 19958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "92bad131b5a04a85bcab8632eac07cc5"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "287f7ff08dc8662e752d59b5ab486ac3"
server_key_iv: "cacae1091e9ac4be1979f3286cf7ea4d"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.825975 19958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:31.836620 19958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:31.838553 19958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:31.838696 19958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:31.839146 19958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:31.839197 19958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:31.839227 19958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:31.839251 19958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:27:31 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:31.845744 19958 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:36075
I20260925 02:27:31.845814 20091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:36075 every 8 connection(s)
I20260925 02:27:31.846057 19958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:31.847483 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.839459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:33707 (local address 127.11.220.254:43463)
0925 02:27:31.839519 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:31.839521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:31.839701 (+ 180us) server_negotiation.cc:407] Connection header received
0925 02:27:31.840072 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:31.840074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:31.840104 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:31.840190 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:31.840818 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.841278 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.842238 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.842422 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.842495 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.844753 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:31.844755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:31.844755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:31.844762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:31.846211 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.846607 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.846608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.846609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.846664 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.846880 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.846881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.846881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.846986 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:31.847056 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:31.847306 (+ 250us) server_negotiation.cc:299] Negotiation successful
0925 02:27:31.847381 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 02:27:31.847615 19982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.839445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43463 (local address 127.11.220.194:33707)
0925 02:27:31.839580 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:31.839613 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:31.839976 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:31.840235 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:31.840238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:31.840535 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:31.840735 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.840739 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.841374 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.841375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.842160 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.842163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.842505 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.842506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.842615 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.843070 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:31.843078 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:31.844657 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:31.846296 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.846303 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.846304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.846512 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.846717 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.846718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.846719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.846817 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.847048 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:31.847055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:31.847177 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 02:27:31.847272 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 02:27:31.847401 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":79,"threads_started":1}
I20260925 02:27:31.848006 20092 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43463
I20260925 02:27:31.848086 20092 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:31.848250 20092 heartbeater.cc:511] Master 127.11.220.254:43463 requested a full tablet report, sending...
I20260925 02:27:31.848572 19740 ts_manager.cc:194] Registered new tserver with Master: 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194:36075)
I20260925 02:27:31.848938 19740 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:33707
I20260925 02:27:31.853005 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 19958
I20260925 02:27:31.853140 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:31.853355 12147 external_mini_cluster.cc:1428] Setting key 025555daa7e24c045f07739f816240e9
I20260925 02:27:31.853631 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.849002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:33327 (local address 127.11.220.254:43463)
0925 02:27:31.849040 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:31.849042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:31.849181 (+ 139us) server_negotiation.cc:407] Connection header received
0925 02:27:31.849203 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:31.849203 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:31.849213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:31.849265 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:31.849554 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.849948 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.850504 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.850653 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.850699 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.851395 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:31.851396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:31.851396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:31.851402 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:31.852431 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.852748 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.852749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.852749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.852831 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.853021 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.853022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.853022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.853226 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:31.853312 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:31.853450 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 02:27:31.853523 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:27:31.853663 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.848927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43463 (local address 127.11.220.194:33327)
0925 02:27:31.849117 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:31.849131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:31.849171 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:31.849266 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:31.849267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:31.849443 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:31.849491 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.849492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.850024 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.850024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.850444 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.850445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.850716 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.850717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.850775 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.851083 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:31.851087 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:31.851324 (+ 237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:31.852540 (+ 1216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.852541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.852541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.852693 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.852906 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.852907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.852907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.852950 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.853303 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:31.853304 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:31.853340 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 02:27:31.853422 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 02:27:31.853537 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for oryx/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab.
Entry for principal oryx/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab.
I20260925 02:27:31.875262 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43463
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab
--principal=oryx/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:31.950409 20101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:31.950584 20101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:31.950606 20101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:31.951589 20101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:31.951619 20101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:31.951637 20101 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 02:27:31.951653 20101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:31.953095 20101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab
--principal=oryx/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.220.254:43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:31.953269 20101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:31.953424 20101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:31.957968 20101 server_base.cc:1034] running on GCE node
W20260925 02:27:31.957959 20109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:31.957930 20107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:31.957940 20106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:31.958312 20101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:31.958554 20101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:31.959714 20101 hybrid_clock.cc:648] HybridClock initialized: now 1790303251959703 us; error 24 us; skew 500 ppm
Sep 25 02:27:31 dist-test-slave-dn4n krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:31.962073 20101 init.cc:377] Logged in from keytab as oryx/127.11.220.195@KRBTEST.COM (short username oryx)
I20260925 02:27:31.962510 20101 webserver.cc:460] Webserver started at http://127.11.220.195:32833/ using document root <none> and password file <none>
I20260925 02:27:31.962738 20101 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:31.962781 20101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:31.962931 20101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:31.963836 20101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "9017aaf9c20f484d9559392272635a5b"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "77a0a555ab9d9e7aa5bb5ca33e7cde83"
server_key_iv: "706b059c9ad1818d49fd05d34cd5620a"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.964196 20101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "9017aaf9c20f484d9559392272635a5b"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "77a0a555ab9d9e7aa5bb5ca33e7cde83"
server_key_iv: "706b059c9ad1818d49fd05d34cd5620a"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.965523 20101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:31.966146 20116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:31.966300 20101 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:31.966372 20101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "9017aaf9c20f484d9559392272635a5b"
format_stamp: "Formatted at 2026-09-25 02:27:31 on dist-test-slave-dn4n"
server_key: "77a0a555ab9d9e7aa5bb5ca33e7cde83"
server_key_iv: "706b059c9ad1818d49fd05d34cd5620a"
server_key_version: "encryptionkey@0"
I20260925 02:27:31.966418 20101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:31.975049 20101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:31.976847 20101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:31.976936 20101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:31.977280 20101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:31.977313 20101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:31.977352 20101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:31.977381 20101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 02:27:31 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:31.983875 20101 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:39513
I20260925 02:27:31.983935 20233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:39513 every 8 connection(s)
I20260925 02:27:31.984192 20101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:31.986591 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.977697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:36849 (local address 127.11.220.254:43463)
0925 02:27:31.977781 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:31.977784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:31.977955 (+ 171us) server_negotiation.cc:407] Connection header received
0925 02:27:31.978319 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:31.978321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:31.978333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:31.978409 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:31.979123 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.979868 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.981279 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.981600 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.981685 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.984050 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:31.984051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:31.984052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:31.984059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:31.985318 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.985721 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.985722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.985722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.985772 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.985936 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.985937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.985937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.986029 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:31.986076 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:31.986419 (+ 343us) server_negotiation.cc:299] Negotiation successful
0925 02:27:31.986487 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 02:27:31.986748 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.977637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43463 (local address 127.11.220.195:36849)
0925 02:27:31.977859 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:31.977881 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:31.978240 (+ 359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:31.978459 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:31.978461 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:31.978797 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:31.979033 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.979039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.979954 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.979955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.981176 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.981179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.981703 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.981705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.981816 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.982240 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:31.982249 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:31.983957 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:31.985376 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.985382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.985384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.985658 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.985811 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.985812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.985812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.985891 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.986151 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:31.986157 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:31.986257 (+ 100us) client_negotiation.cc:769] Sending connection context
0925 02:27:31.986379 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 02:27:31.986485 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":59,"threads_started":1}
I20260925 02:27:31.987196 20234 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43463
I20260925 02:27:31.987272 20234 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:31.987435 20234 heartbeater.cc:511] Master 127.11.220.254:43463 requested a full tablet report, sending...
I20260925 02:27:31.987828 19741 ts_manager.cc:194] Registered new tserver with Master: 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513)
I20260925 02:27:31.988263 19741 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:36849
I20260925 02:27:31.989053 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 20101
I20260925 02:27:31.989149 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:31.989313 12147 external_mini_cluster.cc:1428] Setting key 5d8a8f7f81b7b4508f9176891456f4a9
I20260925 02:27:31.990141 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:27:31.993520 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.988189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:36977 (local address 127.11.220.254:43463)
0925 02:27:31.988226 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:31.988227 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:31.988338 (+ 111us) server_negotiation.cc:407] Connection header received
0925 02:27:31.988408 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:31.988409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:31.988417 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:31.988482 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:31.988793 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.989404 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.990231 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.990457 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.990517 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.991257 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:31.991259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:31.991259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:31.991266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:31.992570 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.992932 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.992932 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.992933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.992975 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:31.993158 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:31.993159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:31.993159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:31.993240 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:31.993285 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:31.993425 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 02:27:31.993471 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:27:31.993635 20241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:31.988128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43463 (local address 127.11.220.195:36977)
0925 02:27:31.988280 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:31.988290 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:31.988361 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:31.988487 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:31.988488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:31.988666 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:31.988721 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.988723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.989498 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.989499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.990147 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:31.990149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:31.990522 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:31.990523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:31.990566 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:31.990940 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:31.990945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:31.991180 (+ 235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:31.992639 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.992640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.992641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.992866 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.993028 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:31.993029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:31.993029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:31.993098 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:31.993298 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:31.993299 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:31.993342 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 02:27:31.993411 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:27:31.993509 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":63,"threads_started":1}
Sep 25 02:27:31 dist-test-slave-dn4n krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 02:27:32.009824 19809 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:32.005226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52914 (local address 127.11.220.254:43463)
0925 02:27:32.005277 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.005278 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.005286 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:32.005302 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.005303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.005310 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.005358 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:32.005861 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.006267 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.006862 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.007012 (+ 150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.007055 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.009734 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52914: BlockingRecv error: recv got EOF from 127.0.0.1:52914 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:32.015978 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.010463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52922 (local address 127.11.220.254:43463)
0925 02:27:32.010508 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.010510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.010591 (+ 81us) server_negotiation.cc:407] Connection header received
0925 02:27:32.010631 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.010631 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.010638 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.010688 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:32.011044 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.011423 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.011968 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.012114 (+ 146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.012154 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.013855 (+ 1701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:32.013856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:32.013856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:32.013861 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:32.015197 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:32.015459 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:32.015459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:32.015459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:32.015503 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:32.015631 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:32.015631 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:32.015631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:32.015703 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:32.015765 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.015863 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.015929 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 02:27:32.017429 19741 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:32.018850 19741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:32.024648 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.020745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:39513 (local address 127.0.0.1:51038)
0925 02:27:32.021077 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.021110 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.021205 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.021422 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.021425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.021426 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:32.021475 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.021476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.022618 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.022619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.023341 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.023343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.024285 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.024286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.024364 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.024376 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.024503 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.024574 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":44,"threads_started":1}
I20260925 02:27:32.024683 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.020889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.11.220.195:39513)
0925 02:27:32.021042 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.021045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.021189 (+ 144us) server_negotiation.cc:407] Connection header received
0925 02:27:32.021276 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.021278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.021293 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.021360 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:32.021568 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.022542 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.023513 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.024197 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.024291 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.024348 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.024529 (+ 181us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.024573 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260925 02:27:32.025012 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.020914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:36075 (local address 127.0.0.1:59748)
0925 02:27:32.021310 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.021319 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.021362 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.021661 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.021669 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.021671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:32.021702 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.021704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.022829 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.022830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.023791 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.023794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.024617 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.024618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.024682 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.024686 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.024890 (+ 204us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.024924 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":87,"threads_started":1}
I20260925 02:27:32.025162 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.020824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.11.220.193:41993)
0925 02:27:32.021021 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.021026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.021273 (+ 247us) server_negotiation.cc:407] Connection header received
0925 02:27:32.021339 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.021341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.021356 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.021423 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:32.021607 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.022290 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.023457 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.024212 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.024296 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.024348 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.024995 (+ 647us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.025044 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":79,"threads_started":1}
I20260925 02:27:32.025219 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.020746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:41993 (local address 127.0.0.1:46654)
0925 02:27:32.021210 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.021219 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.021272 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.021499 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.021499 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.021501 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:32.021541 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.021543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.022372 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.022373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.023342 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.023344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.024834 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.024835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.024879 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.024881 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.024958 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.024995 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":3,"thread_start_us":44,"threads_started":1}
I20260925 02:27:32.025362 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.021020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.11.220.194:36075)
0925 02:27:32.021188 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.021192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.021359 (+ 167us) server_negotiation.cc:407] Connection header received
0925 02:27:32.021422 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.021424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.021436 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.021504 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:32.021811 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.022754 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.023873 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.024542 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.024627 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.024682 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.024931 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.024981 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260925 02:27:32.025779 19880 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1b88bcba6641d6ba5b98c8c1133f50 (DEFAULT_TABLE table=test-table [id=a0ba74bb16034d13a2939d8d911d555d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.025753 20166 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1b88bcba6641d6ba5b98c8c1133f50 (DEFAULT_TABLE table=test-table [id=a0ba74bb16034d13a2939d8d911d555d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.026089 19880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1b88bcba6641d6ba5b98c8c1133f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.026089 20166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1b88bcba6641d6ba5b98c8c1133f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.025995 20023 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1b88bcba6641d6ba5b98c8c1133f50 (DEFAULT_TABLE table=test-table [id=a0ba74bb16034d13a2939d8d911d555d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.026247 20023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1b88bcba6641d6ba5b98c8c1133f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.029345 20260 tablet_bootstrap.cc:492] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: Bootstrap starting.
I20260925 02:27:32.029506 20261 tablet_bootstrap.cc:492] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: Bootstrap starting.
I20260925 02:27:32.029855 20262 tablet_bootstrap.cc:492] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: Bootstrap starting.
I20260925 02:27:32.029973 20260 tablet_bootstrap.cc:654] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.030076 20261 tablet_bootstrap.cc:654] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.030287 20260 log.cc:824] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:32.030330 20261 log.cc:824] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:32.030460 20262 tablet_bootstrap.cc:654] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.030794 20262 log.cc:824] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:32.031011 20260 tablet_bootstrap.cc:492] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: No bootstrap required, opened a new log
I20260925 02:27:32.031023 20261 tablet_bootstrap.cc:492] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: No bootstrap required, opened a new log
I20260925 02:27:32.031078 20260 ts_tablet_manager.cc:1397] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:32.031078 20261 ts_tablet_manager.cc:1397] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:32.031462 20262 tablet_bootstrap.cc:492] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: No bootstrap required, opened a new log
I20260925 02:27:32.031535 20262 ts_tablet_manager.cc:1397] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:32.032915 20261 raft_consensus.cc:348] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.032915 20260 raft_consensus.cc:348] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.032956 20262 raft_consensus.cc:348] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.033064 20260 raft_consensus.cc:374] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.033064 20261 raft_consensus.cc:374] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.033082 20262 raft_consensus.cc:374] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.033098 20261 raft_consensus.cc:729] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9017aaf9c20f484d9559392272635a5b, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.033098 20260 raft_consensus.cc:729] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92bad131b5a04a85bcab8632eac07cc5, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.033135 20262 raft_consensus.cc:729] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c3f96980d44346847183a86a96cc25, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.033227 20260 consensus_queue.cc:260] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.033243 20262 consensus_queue.cc:260] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.033262 20261 consensus_queue.cc:260] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.033471 20262 ts_tablet_manager.cc:1428] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:27:32.033473 20260 ts_tablet_manager.cc:1428] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:27:32.033502 20261 ts_tablet_manager.cc:1428] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:27:32.033598 20092 heartbeater.cc:503] Master 127.11.220.254:43463 was elected leader, sending a full tablet report...
I20260925 02:27:32.033596 19949 heartbeater.cc:503] Master 127.11.220.254:43463 was elected leader, sending a full tablet report...
I20260925 02:27:32.033849 20234 heartbeater.cc:503] Master 127.11.220.254:43463 was elected leader, sending a full tablet report...
W20260925 02:27:32.096699 20093 tablet.cc:2369] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:32.120992 20266 raft_consensus.cc:482] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:32.121074 20266 raft_consensus.cc:504] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.121372 20266 leader_election.cc:288] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993), 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513)
I20260925 02:27:32.123997 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.121599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:41375 (local address 127.11.220.193:41993)
0925 02:27:32.121671 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.121673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.121683 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:27:32.121711 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.121712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.121724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.121785 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:32.121954 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.122430 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.123323 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.123658 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.123707 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.123714 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.123892 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.123931 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:27:32.124002 19982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.121541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:41993 (local address 127.11.220.194:41375)
0925 02:27:32.121593 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.121603 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.121647 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.121806 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.121807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.121809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:32.121880 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.121882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.122502 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.122502 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.123236 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.123239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.123733 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.123734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.123817 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.123822 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.123864 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.123910 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 02:27:32.124050 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.121633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:33403 (local address 127.11.220.195:39513)
0925 02:27:32.121684 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.121686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.121792 (+ 106us) server_negotiation.cc:407] Connection header received
0925 02:27:32.121820 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.121821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.121830 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.121887 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:32.122075 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.122508 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.123357 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.123680 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.123765 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.123773 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.123928 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.123966 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:27:32.124106 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.121559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:39513 (local address 127.11.220.194:33403)
0925 02:27:32.121710 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.121719 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.121776 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.121898 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.121898 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.121900 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:32.121991 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.121993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.122581 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.122581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.123252 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.123254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.123779 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.123780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.123837 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.123840 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.123908 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.123941 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260925 02:27:32.124402 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1b88bcba6641d6ba5b98c8c1133f50" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9017aaf9c20f484d9559392272635a5b" is_pre_election: true
I20260925 02:27:32.124486 19900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1b88bcba6641d6ba5b98c8c1133f50" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c3f96980d44346847183a86a96cc25" is_pre_election: true
I20260925 02:27:32.124567 20187 raft_consensus.cc:2454] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 0.
I20260925 02:27:32.124648 19900 raft_consensus.cc:2454] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 0.
I20260925 02:27:32.124781 19976 leader_election.cc:302] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9017aaf9c20f484d9559392272635a5b, 92bad131b5a04a85bcab8632eac07cc5; no voters:
I20260925 02:27:32.124924 20266 raft_consensus.cc:2781] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:32.124995 20266 raft_consensus.cc:482] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:32.125028 20266 raft_consensus.cc:3037] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.125890 20266 raft_consensus.cc:504] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.126030 20266 leader_election.cc:288] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 election: Requested vote from peers 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993), 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513)
I20260925 02:27:32.126161 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1b88bcba6641d6ba5b98c8c1133f50" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9017aaf9c20f484d9559392272635a5b"
I20260925 02:27:32.126175 19900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1b88bcba6641d6ba5b98c8c1133f50" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c3f96980d44346847183a86a96cc25"
I20260925 02:27:32.126250 19900 raft_consensus.cc:3037] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.126255 20187 raft_consensus.cc:3037] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.126881 20187 raft_consensus.cc:2454] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 1.
I20260925 02:27:32.126950 19900 raft_consensus.cc:2454] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 1.
I20260925 02:27:32.127076 19976 leader_election.cc:302] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9017aaf9c20f484d9559392272635a5b, 92bad131b5a04a85bcab8632eac07cc5; no voters:
I20260925 02:27:32.127156 20266 raft_consensus.cc:2781] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:32.127287 20266 raft_consensus.cc:686] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 1 LEADER]: Becoming Leader. State: Replica: 92bad131b5a04a85bcab8632eac07cc5, State: Running, Role: LEADER
I20260925 02:27:32.127379 20266 consensus_queue.cc:237] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.128091 19740 catalog_manager.cc:5495] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194). New cstate: current_term: 1 leader_uuid: "92bad131b5a04a85bcab8632eac07cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } health_report { overall_health: UNKNOWN } } }
I20260925 02:27:32.137656 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.135798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.11.220.194:36075)
0925 02:27:32.135828 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.135830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.135843 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:27:32.135896 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.135897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.135909 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.135960 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:32.136052 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.136443 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.137121 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.137272 (+ 151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.137314 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.137436 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:32.137510 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:32.137558 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.137569 (+ 11us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.137582 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 02:27:32.139037 19900 raft_consensus.cc:1261] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Refusing update from remote peer 92bad131b5a04a85bcab8632eac07cc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:32.139037 20187 raft_consensus.cc:1261] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Refusing update from remote peer 92bad131b5a04a85bcab8632eac07cc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:32.139371 20270 consensus_queue.cc:1035] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:32.139504 20266 consensus_queue.cc:1035] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:32.141631 20273 mvcc.cc:205] Tried to move back new op lower bound from 7333082120759156736 to 7333082120714244096. Current Snapshot: MvccSnapshot[applied={T|T < 7333082120759156736}]
I20260925 02:27:32.142493 20275 mvcc.cc:205] Tried to move back new op lower bound from 7333082120759156736 to 7333082120714244096. Current Snapshot: MvccSnapshot[applied={T|T < 7333082120759156736}]
I20260925 02:27:32.144017 19740 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:32.144120 19740 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:32.144801 19740 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9: Sending DeleteTablet for 3 replicas of tablet 4a1b88bcba6641d6ba5b98c8c1133f50
I20260925 02:27:32.145113 20166 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1b88bcba6641d6ba5b98c8c1133f50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:51038
I20260925 02:27:32.145119 19880 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1b88bcba6641d6ba5b98c8c1133f50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:46654
I20260925 02:27:32.145236 19740 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:32.145380 19740 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:32.145417 20023 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1b88bcba6641d6ba5b98c8c1133f50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:59748
I20260925 02:27:32.145653 20284 tablet_replica.cc:330] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: stopping tablet replica
I20260925 02:27:32.145776 20284 raft_consensus.cc:2229] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:32.145877 20284 raft_consensus.cc:2258] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.146095 20283 tablet_replica.cc:330] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: stopping tablet replica
I20260925 02:27:32.146196 20283 raft_consensus.cc:2229] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:32.146296 20283 raft_consensus.cc:2258] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.146587 20284 ts_tablet_manager.cc:1905] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.146680 20166 tablet_service.cc:1467] Processing CreateTablet for tablet 02a5a512f9f642eabee5e61cd8b83adf (DEFAULT_TABLE table=test-table [id=26bacecbb50c4152828a06516a4fcf6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.146735 19880 tablet_service.cc:1467] Processing CreateTablet for tablet 02a5a512f9f642eabee5e61cd8b83adf (DEFAULT_TABLE table=test-table [id=26bacecbb50c4152828a06516a4fcf6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.146780 20166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a5a512f9f642eabee5e61cd8b83adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.146816 19880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a5a512f9f642eabee5e61cd8b83adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.147913 20262 tablet_bootstrap.cc:492] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: Bootstrap starting.
I20260925 02:27:32.148133 20261 tablet_bootstrap.cc:492] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: Bootstrap starting.
I20260925 02:27:32.148214 20283 ts_tablet_manager.cc:1905] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.148294 20262 tablet_bootstrap.cc:654] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.148542 20261 tablet_bootstrap.cc:654] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.148940 20023 tablet_service.cc:1467] Processing CreateTablet for tablet 02a5a512f9f642eabee5e61cd8b83adf (DEFAULT_TABLE table=test-table [id=26bacecbb50c4152828a06516a4fcf6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.149053 20023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a5a512f9f642eabee5e61cd8b83adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.149175 20262 tablet_bootstrap.cc:492] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: No bootstrap required, opened a new log
I20260925 02:27:32.149194 20284 ts_tablet_manager.cc:1918] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:32.149214 20262 ts_tablet_manager.cc:1397] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:32.149250 20284 log.cc:1196] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/4a1b88bcba6641d6ba5b98c8c1133f50
I20260925 02:27:32.149343 20262 raft_consensus.cc:348] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.149410 20262 raft_consensus.cc:374] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.149427 20262 raft_consensus.cc:729] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c3f96980d44346847183a86a96cc25, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.149488 20284 ts_tablet_manager.cc:1939] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 80c3f96980d44346847183a86a96cc25: Deleting consensus metadata
I20260925 02:27:32.149466 20262 consensus_queue.cc:260] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.149560 20262 ts_tablet_manager.cc:1428] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:32.149596 20283 ts_tablet_manager.cc:1918] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:32.149652 20283 log.cc:1196] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/4a1b88bcba6641d6ba5b98c8c1133f50
I20260925 02:27:32.149839 19728 catalog_manager.cc:4841] TS 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993): tablet 4a1b88bcba6641d6ba5b98c8c1133f50 (table test-table [id=a0ba74bb16034d13a2939d8d911d555d]) successfully deleted
I20260925 02:27:32.149884 20283 ts_tablet_manager.cc:1939] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 9017aaf9c20f484d9559392272635a5b: Deleting consensus metadata
I20260925 02:27:32.150370 20260 tablet_bootstrap.cc:492] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5: Bootstrap starting.
I20260925 02:27:32.150424 19727 catalog_manager.cc:4841] TS 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513): tablet 4a1b88bcba6641d6ba5b98c8c1133f50 (table test-table [id=a0ba74bb16034d13a2939d8d911d555d]) successfully deleted
I20260925 02:27:32.150736 20285 tablet_replica.cc:330] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: stopping tablet replica
I20260925 02:27:32.150756 20260 tablet_bootstrap.cc:654] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.150794 20261 tablet_bootstrap.cc:492] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: No bootstrap required, opened a new log
I20260925 02:27:32.150816 20285 raft_consensus.cc:2229] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:32.150830 20261 ts_tablet_manager.cc:1397] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260925 02:27:32.150911 20285 raft_consensus.cc:2258] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.150957 20261 raft_consensus.cc:348] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.151018 20261 raft_consensus.cc:374] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.151036 20261 raft_consensus.cc:729] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9017aaf9c20f484d9559392272635a5b, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.151070 20261 consensus_queue.cc:260] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.151140 20261 ts_tablet_manager.cc:1428] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:32.151273 20285 ts_tablet_manager.cc:1905] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.152506 20285 ts_tablet_manager.cc:1918] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:32.152572 20285 log.cc:1196] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/4a1b88bcba6641d6ba5b98c8c1133f50
I20260925 02:27:32.152805 20285 ts_tablet_manager.cc:1939] T 4a1b88bcba6641d6ba5b98c8c1133f50 P 92bad131b5a04a85bcab8632eac07cc5: Deleting consensus metadata
I20260925 02:27:32.152822 20260 tablet_bootstrap.cc:492] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5: No bootstrap required, opened a new log
I20260925 02:27:32.152874 20260 ts_tablet_manager.cc:1397] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 02:27:32.153005 20260 raft_consensus.cc:348] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.153072 20260 raft_consensus.cc:374] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.153095 20260 raft_consensus.cc:729] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92bad131b5a04a85bcab8632eac07cc5, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.153128 20260 consensus_queue.cc:260] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.153213 20260 ts_tablet_manager.cc:1428] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:32.153380 19728 catalog_manager.cc:4841] TS 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194:36075): tablet 4a1b88bcba6641d6ba5b98c8c1133f50 (table test-table [id=a0ba74bb16034d13a2939d8d911d555d]) successfully deleted
W20260925 02:27:32.212536 19950 tablet.cc:2369] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:27:32.234774 20235 tablet.cc:2369] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:32.245626 20280 raft_consensus.cc:482] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:32.245694 20280 raft_consensus.cc:504] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.245852 20280 leader_election.cc:288] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993), 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513)
I20260925 02:27:32.246098 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a5a512f9f642eabee5e61cd8b83adf" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9017aaf9c20f484d9559392272635a5b" is_pre_election: true
I20260925 02:27:32.246119 19900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a5a512f9f642eabee5e61cd8b83adf" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c3f96980d44346847183a86a96cc25" is_pre_election: true
I20260925 02:27:32.246203 20187 raft_consensus.cc:2454] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 0.
I20260925 02:27:32.246203 19900 raft_consensus.cc:2454] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 0.
I20260925 02:27:32.246340 19977 leader_election.cc:302] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80c3f96980d44346847183a86a96cc25, 92bad131b5a04a85bcab8632eac07cc5; no voters:
I20260925 02:27:32.246460 20280 raft_consensus.cc:2781] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:32.246527 20280 raft_consensus.cc:482] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:32.246557 20280 raft_consensus.cc:3037] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.247181 20280 raft_consensus.cc:504] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.247313 20280 leader_election.cc:288] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 election: Requested vote from peers 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993), 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513)
I20260925 02:27:32.247473 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a5a512f9f642eabee5e61cd8b83adf" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9017aaf9c20f484d9559392272635a5b"
I20260925 02:27:32.247471 19900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a5a512f9f642eabee5e61cd8b83adf" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c3f96980d44346847183a86a96cc25"
I20260925 02:27:32.247542 20187 raft_consensus.cc:3037] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.247561 19900 raft_consensus.cc:3037] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.248167 20187 raft_consensus.cc:2454] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 1.
I20260925 02:27:32.248311 19976 leader_election.cc:302] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9017aaf9c20f484d9559392272635a5b, 92bad131b5a04a85bcab8632eac07cc5; no voters:
I20260925 02:27:32.248353 19900 raft_consensus.cc:2454] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 1.
I20260925 02:27:32.248466 20280 raft_consensus.cc:2781] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:32.248513 20280 raft_consensus.cc:686] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 1 LEADER]: Becoming Leader. State: Replica: 92bad131b5a04a85bcab8632eac07cc5, State: Running, Role: LEADER
I20260925 02:27:32.248564 20280 consensus_queue.cc:237] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.249118 19740 catalog_manager.cc:5495] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194). New cstate: current_term: 1 leader_uuid: "92bad131b5a04a85bcab8632eac07cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } health_report { overall_health: UNKNOWN } } }
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303251, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.254@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:32.294675 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.289042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43463 (local address 127.0.0.1:52924)
0925 02:27:32.289209 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.289218 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.289271 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.289386 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.289387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.289660 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:32.289704 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.289705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.290253 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.290254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.290717 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.290718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.290982 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.290983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.291027 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.291457 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:32.291461 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:32.292665 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:32.293819 (+ 1154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:32.293825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:32.293827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:32.294002 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:32.294165 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:32.294165 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:32.294166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:32.294209 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:32.294397 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:32.294403 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:32.294439 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.294491 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.294584 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
I20260925 02:27:32.294732 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.289088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.11.220.254:43463)
0925 02:27:32.289152 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.289153 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.289269 (+ 116us) server_negotiation.cc:407] Connection header received
0925 02:27:32.289309 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.289310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.289317 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.289367 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:32.289760 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.290196 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.290774 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.290928 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.290970 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.292733 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:32.292734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:32.292734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:32.292740 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:32.293758 (+ 1018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:32.294082 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:32.294083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:32.294083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:32.294128 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:32.294254 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:32.294255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:32.294255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:32.294342 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:32.294389 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.294521 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.294625 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":34}
I20260925 02:27:32.296406 19740 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:52924:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 02:27:32.296658 19740 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:32.297960 19880 tablet_service.cc:1467] Processing CreateTablet for tablet f84a564bf26740d8909d423e287332ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fb643a8e671497b9d5bf26a716fb0ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:27:32.297986 20023 tablet_service.cc:1467] Processing CreateTablet for tablet f84a564bf26740d8909d423e287332ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fb643a8e671497b9d5bf26a716fb0ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:27:32.297983 20166 tablet_service.cc:1467] Processing CreateTablet for tablet f84a564bf26740d8909d423e287332ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5fb643a8e671497b9d5bf26a716fb0ca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 02:27:32.298087 19880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84a564bf26740d8909d423e287332ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.298111 20023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84a564bf26740d8909d423e287332ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.298110 20166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f84a564bf26740d8909d423e287332ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.299037 20262 tablet_bootstrap.cc:492] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: Bootstrap starting.
I20260925 02:27:32.299448 20262 tablet_bootstrap.cc:654] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.299471 20260 tablet_bootstrap.cc:492] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: Bootstrap starting.
I20260925 02:27:32.299783 20260 tablet_bootstrap.cc:654] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.300047 20261 tablet_bootstrap.cc:492] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: Bootstrap starting.
I20260925 02:27:32.300175 20262 tablet_bootstrap.cc:492] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: No bootstrap required, opened a new log
I20260925 02:27:32.300230 20262 ts_tablet_manager.cc:1397] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:32.300377 20262 raft_consensus.cc:348] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.300483 20262 raft_consensus.cc:374] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.300504 20261 tablet_bootstrap.cc:654] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.300534 20262 raft_consensus.cc:729] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c3f96980d44346847183a86a96cc25, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.300591 20262 consensus_queue.cc:260] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.300719 20262 ts_tablet_manager.cc:1428] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:32.300796 20267 tablet_replica.cc:439] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } } }
I20260925 02:27:32.300932 20260 tablet_bootstrap.cc:492] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: No bootstrap required, opened a new log
I20260925 02:27:32.300922 20267 tablet_replica.cc:442] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:32.300973 20260 ts_tablet_manager.cc:1397] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:32.301046 20260 raft_consensus.cc:348] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.301086 20260 raft_consensus.cc:374] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.301105 20260 raft_consensus.cc:729] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92bad131b5a04a85bcab8632eac07cc5, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.301159 20260 consensus_queue.cc:260] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.301260 20260 ts_tablet_manager.cc:1428] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:32.301280 20266 tablet_replica.cc:439] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } } }
I20260925 02:27:32.301342 20266 tablet_replica.cc:442] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:32.301640 20261 tablet_bootstrap.cc:492] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: No bootstrap required, opened a new log
I20260925 02:27:32.301698 20261 ts_tablet_manager.cc:1397] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 02:27:32.301847 20261 raft_consensus.cc:348] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.301911 20261 raft_consensus.cc:374] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.301940 20261 raft_consensus.cc:729] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9017aaf9c20f484d9559392272635a5b, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.301975 20261 consensus_queue.cc:260] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.302052 20261 ts_tablet_manager.cc:1428] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:32.302130 20268 tablet_replica.cc:439] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } } }
I20260925 02:27:32.302249 20268 tablet_replica.cc:442] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:32.345263 20268 raft_consensus.cc:482] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:32.345327 20268 raft_consensus.cc:504] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.345593 20268 leader_election.cc:288] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993), 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194:36075)
I20260925 02:27:32.348315 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.345807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:40901 (local address 127.11.220.193:41993)
0925 02:27:32.345864 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.345866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.345890 (+ 24us) server_negotiation.cc:407] Connection header received
0925 02:27:32.345912 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.345913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.345923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.345965 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:32.346175 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.346663 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.347631 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.348038 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.348083 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.348090 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.348243 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.348270 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:27:32.348322 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.345737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:41993 (local address 127.11.220.195:40901)
0925 02:27:32.345795 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.345808 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.345855 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.346011 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.346027 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.346029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:32.346102 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.346104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.346737 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.346738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.347560 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.347563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.348100 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.348101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.348176 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.348180 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.348225 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.348259 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 02:27:32.348644 19900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84a564bf26740d8909d423e287332ec" candidate_uuid: "9017aaf9c20f484d9559392272635a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c3f96980d44346847183a86a96cc25" is_pre_election: true
I20260925 02:27:32.348739 19900 raft_consensus.cc:2454] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9017aaf9c20f484d9559392272635a5b in term 0.
I20260925 02:27:32.348809 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.345841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:36075 (local address 127.11.220.195:46293)
0925 02:27:32.345968 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.345983 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.346029 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.346157 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.346158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.346160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:32.346237 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.346238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.346788 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.346789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.347847 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.347849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.348550 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.348551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.348634 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.348637 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.348705 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.348740 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":88,"threads_started":1}
I20260925 02:27:32.348800 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.345895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:46293 (local address 127.11.220.194:36075)
0925 02:27:32.345942 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.345944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.346037 (+ 93us) server_negotiation.cc:407] Connection header received
0925 02:27:32.346069 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.346070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.346083 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.346130 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:32.346297 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.346716 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.347946 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.348402 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.348502 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.348510 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.348710 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.348746 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 02:27:32.348922 20120 leader_election.cc:302] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80c3f96980d44346847183a86a96cc25, 9017aaf9c20f484d9559392272635a5b; no voters:
I20260925 02:27:32.349044 20268 raft_consensus.cc:2781] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:32.349094 20268 raft_consensus.cc:482] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:32.349135 20268 raft_consensus.cc:3037] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.349159 20045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84a564bf26740d8909d423e287332ec" candidate_uuid: "9017aaf9c20f484d9559392272635a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92bad131b5a04a85bcab8632eac07cc5" is_pre_election: true
I20260925 02:27:32.349284 20045 raft_consensus.cc:2454] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9017aaf9c20f484d9559392272635a5b in term 0.
I20260925 02:27:32.349843 20268 raft_consensus.cc:504] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.349931 20268 leader_election.cc:288] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [CANDIDATE]: Term 1 election: Requested vote from peers 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993), 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194:36075)
I20260925 02:27:32.350060 19900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84a564bf26740d8909d423e287332ec" candidate_uuid: "9017aaf9c20f484d9559392272635a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c3f96980d44346847183a86a96cc25"
I20260925 02:27:32.350143 19900 raft_consensus.cc:3037] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.350123 20045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f84a564bf26740d8909d423e287332ec" candidate_uuid: "9017aaf9c20f484d9559392272635a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92bad131b5a04a85bcab8632eac07cc5"
I20260925 02:27:32.350206 20045 raft_consensus.cc:3037] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.350697 19900 raft_consensus.cc:2454] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9017aaf9c20f484d9559392272635a5b in term 1.
I20260925 02:27:32.350749 20045 raft_consensus.cc:2454] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9017aaf9c20f484d9559392272635a5b in term 1.
I20260925 02:27:32.350874 20120 leader_election.cc:302] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80c3f96980d44346847183a86a96cc25, 9017aaf9c20f484d9559392272635a5b; no voters:
I20260925 02:27:32.350968 20268 raft_consensus.cc:2781] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:32.351083 20268 raft_consensus.cc:686] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [term 1 LEADER]: Becoming Leader. State: Replica: 9017aaf9c20f484d9559392272635a5b, State: Running, Role: LEADER
I20260925 02:27:32.351131 20268 consensus_queue.cc:237] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.351434 20301 tablet_replica.cc:439] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: TxnStatusTablet state changed. Reason: New leader 9017aaf9c20f484d9559392272635a5b. Latest consensus state: current_term: 1 leader_uuid: "9017aaf9c20f484d9559392272635a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } } }
I20260925 02:27:32.351522 20301 tablet_replica.cc:442] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:27:32.351677 20303 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 02:27:32.351800 20303 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 02:27:32.351794 19740 catalog_manager.cc:5495] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9017aaf9c20f484d9559392272635a5b (127.11.220.195). New cstate: current_term: 1 leader_uuid: "9017aaf9c20f484d9559392272635a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } health_report { overall_health: HEALTHY } } }
I20260925 02:27:32.359259 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.357116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.11.220.195:39513)
0925 02:27:32.357158 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.357160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.357168 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:32.357228 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.357229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.357245 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.357296 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:32.357387 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.357804 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.358522 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.358715 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.358782 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.358963 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:32.359059 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:32.359140 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.359190 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.359208 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 02:27:32.359297 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.357038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:39513 (local address 127.0.0.1:51046)
0925 02:27:32.357105 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.357114 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.357158 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.357299 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.357299 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.357300 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:32.357342 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.357344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.357885 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.357886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.358443 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.358445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.358784 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.358785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.358848 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.358864 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:32.359127 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:32.359136 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.359185 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.359228 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 02:27:32.360194 19900 raft_consensus.cc:1261] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Refusing update from remote peer 9017aaf9c20f484d9559392272635a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:32.360282 20045 raft_consensus.cc:1261] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Refusing update from remote peer 9017aaf9c20f484d9559392272635a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:32.360486 20301 consensus_queue.cc:1035] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:32.360584 20268 consensus_queue.cc:1035] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:32.361068 20266 tablet_replica.cc:439] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: TxnStatusTablet state changed. Reason: New leader 9017aaf9c20f484d9559392272635a5b. Latest consensus state: current_term: 1 leader_uuid: "9017aaf9c20f484d9559392272635a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } } }
I20260925 02:27:32.361152 20266 tablet_replica.cc:442] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:32.361364 20267 tablet_replica.cc:439] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: TxnStatusTablet state changed. Reason: New leader 9017aaf9c20f484d9559392272635a5b. Latest consensus state: current_term: 1 leader_uuid: "9017aaf9c20f484d9559392272635a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } } }
I20260925 02:27:32.361433 20267 tablet_replica.cc:442] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:32.361882 20274 mvcc.cc:205] Tried to move back new op lower bound from 7333082121665482752 to 7333082121630658560. Current Snapshot: MvccSnapshot[applied={T|T < 7333082121665482752}]
I20260925 02:27:32.361944 20301 tablet_replica.cc:439] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9017aaf9c20f484d9559392272635a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } } }
I20260925 02:27:32.362026 20301 tablet_replica.cc:442] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:27:32.362072 20301 tablet_replica.cc:439] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9017aaf9c20f484d9559392272635a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } } }
I20260925 02:27:32.362105 20301 tablet_replica.cc:442] T f84a564bf26740d8909d423e287332ec P 9017aaf9c20f484d9559392272635a5b: This TxnStatusTablet replica's current role is: LEADER
I20260925 02:27:32.362830 20267 tablet_replica.cc:439] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9017aaf9c20f484d9559392272635a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } } }
I20260925 02:27:32.362914 20267 tablet_replica.cc:442] T f84a564bf26740d8909d423e287332ec P 80c3f96980d44346847183a86a96cc25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:32.363130 20266 tablet_replica.cc:439] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9017aaf9c20f484d9559392272635a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } } }
I20260925 02:27:32.363209 20266 tablet_replica.cc:442] T f84a564bf26740d8909d423e287332ec P 92bad131b5a04a85bcab8632eac07cc5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 02:27:32.367293 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.365009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:39513 (local address 127.11.220.194:58155)
0925 02:27:32.365112 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.365121 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.365174 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.365279 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.365279 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.365280 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:32.365310 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.365311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.366067 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.366068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.366634 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.366636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.367015 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.367015 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.367047 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.367058 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:32.367189 (+ 131us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:32.367191 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.367220 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.367249 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"thread_start_us":50,"threads_started":1}
I20260925 02:27:32.367290 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.365062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:58155 (local address 127.11.220.195:39513)
0925 02:27:32.365092 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.365094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.365171 (+ 77us) server_negotiation.cc:407] Connection header received
0925 02:27:32.365209 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.365210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.365218 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.365276 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:32.365366 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.365996 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.366704 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.366951 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.367010 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.367116 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:32.367142 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:32.367190 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.367222 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.367236 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:32.368990 20187 raft_consensus.cc:1261] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Refusing update from remote peer 92bad131b5a04a85bcab8632eac07cc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:32.369005 19900 raft_consensus.cc:1261] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Refusing update from remote peer 92bad131b5a04a85bcab8632eac07cc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:32.369166 20280 consensus_queue.cc:1035] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:32.369243 20266 consensus_queue.cc:1035] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:32.376530 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.374060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:56163 (local address 127.11.220.194:36075)
0925 02:27:32.374096 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.374098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.374114 (+ 16us) server_negotiation.cc:407] Connection header received
0925 02:27:32.374182 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.374183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.374190 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.374231 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:32.374364 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.374767 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.375800 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.376072 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.376137 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.376318 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:32.376342 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:32.376375 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.376421 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.376436 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:27:32.376536 20241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.374000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:36075 (local address 127.11.220.195:56163)
0925 02:27:32.374073 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.374084 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:32.374125 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:32.374253 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:32.374254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:32.374255 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 02:27:32.374308 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.374309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.374828 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.374829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.375720 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:32.375723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.376145 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.376146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:32.376224 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.376250 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 02:27:32.376376 (+ 126us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 02:27:32.376377 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 02:27:32.376419 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 02:27:32.376474 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 02:27:32.385128 19741 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:32.385238 19741 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:32.385787 19741 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9: Sending DeleteTablet for 3 replicas of tablet 02a5a512f9f642eabee5e61cd8b83adf
I20260925 02:27:32.386009 20166 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a5a512f9f642eabee5e61cd8b83adf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:51038
I20260925 02:27:32.386020 19880 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a5a512f9f642eabee5e61cd8b83adf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:46654
I20260925 02:27:32.386070 20023 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a5a512f9f642eabee5e61cd8b83adf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:59748
I20260925 02:27:32.386116 20284 tablet_replica.cc:330] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: stopping tablet replica
I20260925 02:27:32.386126 20283 tablet_replica.cc:330] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: stopping tablet replica
I20260925 02:27:32.386137 20285 tablet_replica.cc:330] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5: stopping tablet replica
I20260925 02:27:32.386169 20284 raft_consensus.cc:2229] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:32.386175 20285 raft_consensus.cc:2229] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:32.386173 20283 raft_consensus.cc:2229] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:32.386199 20284 raft_consensus.cc:2258] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.386204 20283 raft_consensus.cc:2258] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.386220 20285 raft_consensus.cc:2258] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.386432 20284 ts_tablet_manager.cc:1905] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.386550 20285 ts_tablet_manager.cc:1905] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.387691 20284 ts_tablet_manager.cc:1918] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:32.387748 20284 log.cc:1196] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/02a5a512f9f642eabee5e61cd8b83adf
I20260925 02:27:32.387987 20284 ts_tablet_manager.cc:1939] T 02a5a512f9f642eabee5e61cd8b83adf P 80c3f96980d44346847183a86a96cc25: Deleting consensus metadata
I20260925 02:27:32.388176 20285 ts_tablet_manager.cc:1918] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:32.388224 20285 log.cc:1196] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/02a5a512f9f642eabee5e61cd8b83adf
I20260925 02:27:32.388283 19728 catalog_manager.cc:4841] TS 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993): tablet 02a5a512f9f642eabee5e61cd8b83adf (table test-table [id=26bacecbb50c4152828a06516a4fcf6f]) successfully deleted
I20260925 02:27:32.388422 20285 ts_tablet_manager.cc:1939] T 02a5a512f9f642eabee5e61cd8b83adf P 92bad131b5a04a85bcab8632eac07cc5: Deleting consensus metadata
I20260925 02:27:32.388955 19728 catalog_manager.cc:4841] TS 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194:36075): tablet 02a5a512f9f642eabee5e61cd8b83adf (table test-table [id=26bacecbb50c4152828a06516a4fcf6f]) successfully deleted
I20260925 02:27:32.389335 20283 ts_tablet_manager.cc:1905] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.390322 20283 ts_tablet_manager.cc:1918] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:32.390363 20283 log.cc:1196] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/02a5a512f9f642eabee5e61cd8b83adf
I20260925 02:27:32.390553 20283 ts_tablet_manager.cc:1939] T 02a5a512f9f642eabee5e61cd8b83adf P 9017aaf9c20f484d9559392272635a5b: Deleting consensus metadata
I20260925 02:27:32.390847 19727 catalog_manager.cc:4841] TS 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513): tablet 02a5a512f9f642eabee5e61cd8b83adf (table test-table [id=26bacecbb50c4152828a06516a4fcf6f]) successfully deleted
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 02:27:32.413676 19809 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:32.408497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52940 (local address 127.11.220.254:43463)
0925 02:27:32.408549 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.408551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.408630 (+ 79us) server_negotiation.cc:407] Connection header received
0925 02:27:32.408707 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.408707 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.408740 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.408814 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:32.409367 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.410139 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.410855 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.411136 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.411199 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.413625 (+ 2426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52940: BlockingRecv error: recv got EOF from 127.0.0.1:52940 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM
I20260925 02:27:32.421175 19809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.414311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52944 (local address 127.11.220.254:43463)
0925 02:27:32.414359 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.414361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.414371 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:27:32.414423 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.414424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.414432 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.414478 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:32.414996 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.415615 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.416321 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.416634 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.416704 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.418912 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:32.418913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:32.418914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:32.418919 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:32.420097 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:32.420506 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:32.420507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:32.420508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:32.420566 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:32.420764 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:32.420765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:32.420765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:32.420855 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:32.420920 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.421065 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.421117 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 02:27:32.422544 19741 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:32.422694 19741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:32.423892 20166 tablet_service.cc:1467] Processing CreateTablet for tablet 32df218496044eb89da90a86b2d8d5b4 (DEFAULT_TABLE table=test-table [id=6579f4efc9954248bbb4917f56a65a82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.423921 20023 tablet_service.cc:1467] Processing CreateTablet for tablet 32df218496044eb89da90a86b2d8d5b4 (DEFAULT_TABLE table=test-table [id=6579f4efc9954248bbb4917f56a65a82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.423950 19880 tablet_service.cc:1467] Processing CreateTablet for tablet 32df218496044eb89da90a86b2d8d5b4 (DEFAULT_TABLE table=test-table [id=6579f4efc9954248bbb4917f56a65a82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.423998 20166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32df218496044eb89da90a86b2d8d5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.424006 20023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32df218496044eb89da90a86b2d8d5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.424059 19880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32df218496044eb89da90a86b2d8d5b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.425190 20262 tablet_bootstrap.cc:492] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25: Bootstrap starting.
I20260925 02:27:32.425194 20261 tablet_bootstrap.cc:492] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b: Bootstrap starting.
I20260925 02:27:32.425452 20261 tablet_bootstrap.cc:654] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.425549 20262 tablet_bootstrap.cc:654] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.425813 20260 tablet_bootstrap.cc:492] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5: Bootstrap starting.
I20260925 02:27:32.426134 20260 tablet_bootstrap.cc:654] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.426136 20261 tablet_bootstrap.cc:492] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b: No bootstrap required, opened a new log
I20260925 02:27:32.426177 20261 ts_tablet_manager.cc:1397] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:32.426317 20261 raft_consensus.cc:348] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.426384 20261 raft_consensus.cc:374] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.426409 20261 raft_consensus.cc:729] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9017aaf9c20f484d9559392272635a5b, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.426434 20261 consensus_queue.cc:260] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.426509 20261 ts_tablet_manager.cc:1428] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:32.426819 20260 tablet_bootstrap.cc:492] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5: No bootstrap required, opened a new log
I20260925 02:27:32.426862 20260 ts_tablet_manager.cc:1397] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:32.426951 20260 raft_consensus.cc:348] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.427004 20260 raft_consensus.cc:374] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.427021 20260 raft_consensus.cc:729] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92bad131b5a04a85bcab8632eac07cc5, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.427080 20260 consensus_queue.cc:260] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.427084 20262 tablet_bootstrap.cc:492] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25: No bootstrap required, opened a new log
I20260925 02:27:32.427131 20262 ts_tablet_manager.cc:1397] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:32.427160 20260 ts_tablet_manager.cc:1428] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:32.427256 20262 raft_consensus.cc:348] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.427316 20262 raft_consensus.cc:374] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.427341 20262 raft_consensus.cc:729] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c3f96980d44346847183a86a96cc25, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.427386 20262 consensus_queue.cc:260] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.427453 20262 ts_tablet_manager.cc:1428] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:32.579209 20317 raft_consensus.cc:482] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:32.579289 20317 raft_consensus.cc:504] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.579460 20317 leader_election.cc:288] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993), 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194:36075)
I20260925 02:27:32.579624 19900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32df218496044eb89da90a86b2d8d5b4" candidate_uuid: "9017aaf9c20f484d9559392272635a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c3f96980d44346847183a86a96cc25" is_pre_election: true
I20260925 02:27:32.579661 20045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32df218496044eb89da90a86b2d8d5b4" candidate_uuid: "9017aaf9c20f484d9559392272635a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92bad131b5a04a85bcab8632eac07cc5" is_pre_election: true
I20260925 02:27:32.579736 19900 raft_consensus.cc:2454] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9017aaf9c20f484d9559392272635a5b in term 0.
I20260925 02:27:32.579742 20045 raft_consensus.cc:2454] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9017aaf9c20f484d9559392272635a5b in term 0.
I20260925 02:27:32.579931 20120 leader_election.cc:302] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80c3f96980d44346847183a86a96cc25, 9017aaf9c20f484d9559392272635a5b; no voters:
I20260925 02:27:32.580049 20317 raft_consensus.cc:2781] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:32.580096 20317 raft_consensus.cc:482] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:32.580135 20317 raft_consensus.cc:3037] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.580806 20317 raft_consensus.cc:504] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.580919 20317 leader_election.cc:288] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [CANDIDATE]: Term 1 election: Requested vote from peers 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993), 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194:36075)
I20260925 02:27:32.581089 20045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32df218496044eb89da90a86b2d8d5b4" candidate_uuid: "9017aaf9c20f484d9559392272635a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92bad131b5a04a85bcab8632eac07cc5"
I20260925 02:27:32.581157 20045 raft_consensus.cc:3037] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.581743 20045 raft_consensus.cc:2454] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9017aaf9c20f484d9559392272635a5b in term 1.
I20260925 02:27:32.581857 19900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32df218496044eb89da90a86b2d8d5b4" candidate_uuid: "9017aaf9c20f484d9559392272635a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c3f96980d44346847183a86a96cc25"
I20260925 02:27:32.581936 19900 raft_consensus.cc:3037] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.581918 20120 leader_election.cc:302] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9017aaf9c20f484d9559392272635a5b, 92bad131b5a04a85bcab8632eac07cc5; no voters:
I20260925 02:27:32.582011 20317 raft_consensus.cc:2781] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:32.582055 20317 raft_consensus.cc:686] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 1 LEADER]: Becoming Leader. State: Replica: 9017aaf9c20f484d9559392272635a5b, State: Running, Role: LEADER
I20260925 02:27:32.582115 20317 consensus_queue.cc:237] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.582564 19740 catalog_manager.cc:5495] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9017aaf9c20f484d9559392272635a5b (127.11.220.195). New cstate: current_term: 1 leader_uuid: "9017aaf9c20f484d9559392272635a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } health_report { overall_health: HEALTHY } } }
I20260925 02:27:32.582729 19900 raft_consensus.cc:2454] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9017aaf9c20f484d9559392272635a5b in term 1.
I20260925 02:27:32.608419 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.606646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51052 (local address 127.11.220.195:39513)
0925 02:27:32.606686 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.606687 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.606714 (+ 27us) server_negotiation.cc:407] Connection header received
0925 02:27:32.606770 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.606770 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.606780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.606810 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:32.606903 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.607316 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.607949 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.608103 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.608146 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.608267 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:32.608291 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:32.608325 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.608362 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.608376 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:27:32.609158 19900 raft_consensus.cc:1261] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Refusing update from remote peer 9017aaf9c20f484d9559392272635a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:32.609153 20045 raft_consensus.cc:1261] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Refusing update from remote peer 9017aaf9c20f484d9559392272635a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:32.609366 20301 consensus_queue.cc:1035] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:32.609440 20317 consensus_queue.cc:1035] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:32.612243 19741 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:32.612305 19741 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:32.612836 19741 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9: Sending DeleteTablet for 3 replicas of tablet 32df218496044eb89da90a86b2d8d5b4
I20260925 02:27:32.613056 20166 tablet_service.cc:1514] Processing DeleteTablet for tablet 32df218496044eb89da90a86b2d8d5b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:51038
I20260925 02:27:32.613062 19880 tablet_service.cc:1514] Processing DeleteTablet for tablet 32df218496044eb89da90a86b2d8d5b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:46654
I20260925 02:27:32.613147 20283 tablet_replica.cc:330] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b: stopping tablet replica
I20260925 02:27:32.613138 20023 tablet_service.cc:1514] Processing DeleteTablet for tablet 32df218496044eb89da90a86b2d8d5b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:59748
I20260925 02:27:32.613200 20283 raft_consensus.cc:2229] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:32.613282 20283 raft_consensus.cc:2258] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.613381 20284 tablet_replica.cc:330] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25: stopping tablet replica
I20260925 02:27:32.613443 20284 raft_consensus.cc:2229] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:32.613484 20284 raft_consensus.cc:2258] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.613547 20285 tablet_replica.cc:330] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5: stopping tablet replica
I20260925 02:27:32.613611 20285 raft_consensus.cc:2229] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:32.613655 20285 raft_consensus.cc:2258] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.613713 20284 ts_tablet_manager.cc:1905] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.613919 20283 ts_tablet_manager.cc:1905] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.613930 20285 ts_tablet_manager.cc:1905] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.615137 20284 ts_tablet_manager.cc:1918] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:32.615187 20284 log.cc:1196] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/32df218496044eb89da90a86b2d8d5b4
I20260925 02:27:32.615286 20283 ts_tablet_manager.cc:1918] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:32.615304 20285 ts_tablet_manager.cc:1918] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:32.615330 20283 log.cc:1196] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/32df218496044eb89da90a86b2d8d5b4
I20260925 02:27:32.615350 20285 log.cc:1196] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/32df218496044eb89da90a86b2d8d5b4
I20260925 02:27:32.615396 20284 ts_tablet_manager.cc:1939] T 32df218496044eb89da90a86b2d8d5b4 P 80c3f96980d44346847183a86a96cc25: Deleting consensus metadata
I20260925 02:27:32.615515 20283 ts_tablet_manager.cc:1939] T 32df218496044eb89da90a86b2d8d5b4 P 9017aaf9c20f484d9559392272635a5b: Deleting consensus metadata
I20260925 02:27:32.615548 20285 ts_tablet_manager.cc:1939] T 32df218496044eb89da90a86b2d8d5b4 P 92bad131b5a04a85bcab8632eac07cc5: Deleting consensus metadata
I20260925 02:27:32.615833 19728 catalog_manager.cc:4841] TS 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993): tablet 32df218496044eb89da90a86b2d8d5b4 (table test-table [id=6579f4efc9954248bbb4917f56a65a82]) successfully deleted
I20260925 02:27:32.615859 19741 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:32.616007 19741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:32.616042 19728 catalog_manager.cc:4841] TS 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194:36075): tablet 32df218496044eb89da90a86b2d8d5b4 (table test-table [id=6579f4efc9954248bbb4917f56a65a82]) successfully deleted
I20260925 02:27:32.616220 19727 catalog_manager.cc:4841] TS 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513): tablet 32df218496044eb89da90a86b2d8d5b4 (table test-table [id=6579f4efc9954248bbb4917f56a65a82]) successfully deleted
I20260925 02:27:32.617172 20166 tablet_service.cc:1467] Processing CreateTablet for tablet 05e5c233a94740e3ab293c6525de0b4b (DEFAULT_TABLE table=test-table [id=37f7921f3bc3441e82f168739c13fab6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.617190 19880 tablet_service.cc:1467] Processing CreateTablet for tablet 05e5c233a94740e3ab293c6525de0b4b (DEFAULT_TABLE table=test-table [id=37f7921f3bc3441e82f168739c13fab6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.617229 20023 tablet_service.cc:1467] Processing CreateTablet for tablet 05e5c233a94740e3ab293c6525de0b4b (DEFAULT_TABLE table=test-table [id=37f7921f3bc3441e82f168739c13fab6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:32.617268 20166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e5c233a94740e3ab293c6525de0b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.617278 19880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e5c233a94740e3ab293c6525de0b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.617304 20023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e5c233a94740e3ab293c6525de0b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:32.618328 20260 tablet_bootstrap.cc:492] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5: Bootstrap starting.
I20260925 02:27:32.618326 20262 tablet_bootstrap.cc:492] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25: Bootstrap starting.
I20260925 02:27:32.618599 20260 tablet_bootstrap.cc:654] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.618706 20262 tablet_bootstrap.cc:654] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.619206 20261 tablet_bootstrap.cc:492] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b: Bootstrap starting.
I20260925 02:27:32.619473 20262 tablet_bootstrap.cc:492] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25: No bootstrap required, opened a new log
I20260925 02:27:32.619511 20261 tablet_bootstrap.cc:654] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:32.619518 20262 ts_tablet_manager.cc:1397] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:32.619659 20262 raft_consensus.cc:348] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.619726 20262 raft_consensus.cc:374] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.619750 20262 raft_consensus.cc:729] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c3f96980d44346847183a86a96cc25, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.619791 20262 consensus_queue.cc:260] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.619876 20262 ts_tablet_manager.cc:1428] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:32.620218 20261 tablet_bootstrap.cc:492] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b: No bootstrap required, opened a new log
I20260925 02:27:32.620265 20261 ts_tablet_manager.cc:1397] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:32.620355 20260 tablet_bootstrap.cc:492] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5: No bootstrap required, opened a new log
I20260925 02:27:32.620352 20261 raft_consensus.cc:348] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.620405 20261 raft_consensus.cc:374] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.620424 20261 raft_consensus.cc:729] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9017aaf9c20f484d9559392272635a5b, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.620417 20260 ts_tablet_manager.cc:1397] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260925 02:27:32.620483 20261 consensus_queue.cc:260] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.620563 20261 ts_tablet_manager.cc:1428] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:32.620558 20260 raft_consensus.cc:348] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.620623 20260 raft_consensus.cc:374] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:32.620646 20260 raft_consensus.cc:729] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92bad131b5a04a85bcab8632eac07cc5, State: Initialized, Role: FOLLOWER
I20260925 02:27:32.620688 20260 consensus_queue.cc:260] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.620769 20260 ts_tablet_manager.cc:1428] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:32.704540 20266 raft_consensus.cc:482] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:32.704635 20266 raft_consensus.cc:504] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.704829 20266 leader_election.cc:288] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993), 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513)
I20260925 02:27:32.705006 19900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e5c233a94740e3ab293c6525de0b4b" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c3f96980d44346847183a86a96cc25" is_pre_election: true
I20260925 02:27:32.705022 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e5c233a94740e3ab293c6525de0b4b" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9017aaf9c20f484d9559392272635a5b" is_pre_election: true
I20260925 02:27:32.705112 20187 raft_consensus.cc:2454] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 0.
I20260925 02:27:32.705118 19900 raft_consensus.cc:2454] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 0.
I20260925 02:27:32.705302 19977 leader_election.cc:302] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80c3f96980d44346847183a86a96cc25, 92bad131b5a04a85bcab8632eac07cc5; no voters:
I20260925 02:27:32.705405 20266 raft_consensus.cc:2781] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:32.705439 20266 raft_consensus.cc:482] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:32.705456 20266 raft_consensus.cc:3037] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.706413 20266 raft_consensus.cc:504] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.706547 20266 leader_election.cc:288] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 election: Requested vote from peers 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993), 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513)
I20260925 02:27:32.706677 19900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e5c233a94740e3ab293c6525de0b4b" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80c3f96980d44346847183a86a96cc25"
I20260925 02:27:32.706707 20187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e5c233a94740e3ab293c6525de0b4b" candidate_uuid: "92bad131b5a04a85bcab8632eac07cc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9017aaf9c20f484d9559392272635a5b"
I20260925 02:27:32.706770 19900 raft_consensus.cc:3037] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.706768 20187 raft_consensus.cc:3037] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:32.707296 19900 raft_consensus.cc:2454] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 1.
I20260925 02:27:32.707304 20187 raft_consensus.cc:2454] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92bad131b5a04a85bcab8632eac07cc5 in term 1.
I20260925 02:27:32.707494 19977 leader_election.cc:302] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80c3f96980d44346847183a86a96cc25, 92bad131b5a04a85bcab8632eac07cc5; no voters:
I20260925 02:27:32.707607 20266 raft_consensus.cc:2781] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:32.707664 20266 raft_consensus.cc:686] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 1 LEADER]: Becoming Leader. State: Replica: 92bad131b5a04a85bcab8632eac07cc5, State: Running, Role: LEADER
I20260925 02:27:32.707715 20266 consensus_queue.cc:237] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } }
I20260925 02:27:32.708258 19740 catalog_manager.cc:5495] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194). New cstate: current_term: 1 leader_uuid: "92bad131b5a04a85bcab8632eac07cc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92bad131b5a04a85bcab8632eac07cc5" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 36075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 } health_report { overall_health: UNKNOWN } } }
I20260925 02:27:32.714532 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.712490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.11.220.194:36075)
0925 02:27:32.712525 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:32.712526 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:32.712533 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:32.712552 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:32.712552 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:32.712562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:32.712635 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:32.712734 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.713185 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.713851 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:32.714043 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:32.714131 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:32.714296 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:32.714328 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:32.714376 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:32.714439 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 02:27:32.714461 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 02:27:32.716619 20187 raft_consensus.cc:1261] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Refusing update from remote peer 92bad131b5a04a85bcab8632eac07cc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:32.716619 19900 raft_consensus.cc:1261] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Refusing update from remote peer 92bad131b5a04a85bcab8632eac07cc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:32.716809 20280 consensus_queue.cc:1035] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9017aaf9c20f484d9559392272635a5b" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 39513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:32.716895 20266 consensus_queue.cc:1035] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80c3f96980d44346847183a86a96cc25" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 41993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:32.725853 19740 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:32.725955 19740 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:32.726470 19740 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93575dcbd5f0463ab2973d0eb36f5eb9: Sending DeleteTablet for 3 replicas of tablet 05e5c233a94740e3ab293c6525de0b4b
I20260925 02:27:32.726696 19880 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e5c233a94740e3ab293c6525de0b4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:46654
I20260925 02:27:32.726694 20166 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e5c233a94740e3ab293c6525de0b4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:51038
I20260925 02:27:32.726708 20023 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e5c233a94740e3ab293c6525de0b4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:32 UTC) from {username='oryx', principal='oryx/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:59748
I20260925 02:27:32.726783 20285 tablet_replica.cc:330] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5: stopping tablet replica
I20260925 02:27:32.726783 20284 tablet_replica.cc:330] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25: stopping tablet replica
I20260925 02:27:32.726831 20284 raft_consensus.cc:2229] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:32.726833 20285 raft_consensus.cc:2229] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:32.726866 20284 raft_consensus.cc:2258] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.726886 20285 raft_consensus.cc:2258] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.726928 20283 tablet_replica.cc:330] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b: stopping tablet replica
I20260925 02:27:32.726998 20283 raft_consensus.cc:2229] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:32.727035 20283 raft_consensus.cc:2258] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:32.727141 20285 ts_tablet_manager.cc:1905] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.727255 20283 ts_tablet_manager.cc:1905] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.727370 20284 ts_tablet_manager.cc:1905] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:32.728421 20285 ts_tablet_manager.cc:1918] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:32.728510 20285 log.cc:1196] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/05e5c233a94740e3ab293c6525de0b4b
I20260925 02:27:32.728624 20283 ts_tablet_manager.cc:1918] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:32.728677 20285 ts_tablet_manager.cc:1939] T 05e5c233a94740e3ab293c6525de0b4b P 92bad131b5a04a85bcab8632eac07cc5: Deleting consensus metadata
I20260925 02:27:32.728674 20283 log.cc:1196] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/05e5c233a94740e3ab293c6525de0b4b
I20260925 02:27:32.728753 20284 ts_tablet_manager.cc:1918] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 02:27:32.728796 20284 log.cc:1196] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/05e5c233a94740e3ab293c6525de0b4b
I20260925 02:27:32.728871 20283 ts_tablet_manager.cc:1939] T 05e5c233a94740e3ab293c6525de0b4b P 9017aaf9c20f484d9559392272635a5b: Deleting consensus metadata
I20260925 02:27:32.728977 20284 ts_tablet_manager.cc:1939] T 05e5c233a94740e3ab293c6525de0b4b P 80c3f96980d44346847183a86a96cc25: Deleting consensus metadata
I20260925 02:27:32.729043 19728 catalog_manager.cc:4841] TS 92bad131b5a04a85bcab8632eac07cc5 (127.11.220.194:36075): tablet 05e5c233a94740e3ab293c6525de0b4b (table test-table [id=37f7921f3bc3441e82f168739c13fab6]) successfully deleted
I20260925 02:27:32.729208 19727 catalog_manager.cc:4841] TS 9017aaf9c20f484d9559392272635a5b (127.11.220.195:39513): tablet 05e5c233a94740e3ab293c6525de0b4b (table test-table [id=37f7921f3bc3441e82f168739c13fab6]) successfully deleted
I20260925 02:27:32.729372 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 19815
W20260925 02:27:32.735268 19728 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:41993 (error 108)
I20260925 02:27:32.735492 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 19958
I20260925 02:27:32.736786 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:32.736620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:41993 (local address 127.0.0.1:46658)
0925 02:27:32.736713 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:32.736758 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:41993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":58,"threads_started":1}
W20260925 02:27:32.736953 19728 catalog_manager.cc:4584] TS 80c3f96980d44346847183a86a96cc25 (127.11.220.193:41993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05e5c233a94740e3ab293c6525de0b4b: Network error: Client connection negotiation failed: client connection to 127.11.220.193:41993: connect: Connection refused (error 111)
I20260925 02:27:32.741505 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 20101
I20260925 02:27:32.747153 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 19708
2026-09-25T02:27:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2388 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[20349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[20349](info): set up 2 sockets
Sep 25 02:27:32 dist-test-slave-dn4n krb5kdc[20349](info): commencing operation
krb5kdc: starting...
W20260925 02:27:33.701715 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:33 dist-test-slave-dn4n krb5kdc[20349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:33.804827 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:36203
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:40217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:36203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:33.879757 20362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:33.879920 20362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:33.879940 20362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:33.880982 20362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:33.881016 20362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:33.881028 20362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:33.881040 20362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:33.881052 20362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:33.882287 20362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:40217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:36203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:36203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:33.882457 20362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:33.882599 20362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:33.887060 20362 server_base.cc:1034] running on GCE node
W20260925 02:27:33.887094 20371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:33.887219 20367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:33.887070 20368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:33.887465 20362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:33.887761 20362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:33.888926 20362 hybrid_clock.cc:648] HybridClock initialized: now 1790303253888909 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T02:27:33Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1143 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:33 dist-test-slave-dn4n krb5kdc[20377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:33 dist-test-slave-dn4n krb5kdc[20377](info): set up 2 sockets
Sep 25 02:27:33 dist-test-slave-dn4n krb5kdc[20377](info): commencing operation
krb5kdc: starting...
W20260925 02:27:34.832957 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:34 dist-test-slave-dn4n krb5kdc[20377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:34.931077 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:40047
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:40047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:35.004405 20390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:35.004643 20390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:35.004680 20390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:35.006165 20390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:35.006213 20390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:35.006235 20390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:35.006250 20390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:35.006268 20390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:35.008186 20390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:40047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:40047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:35.008415 20390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:35.008646 20390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:35.012912 20398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.012914 20396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.012948 20395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:35.013106 20390 server_base.cc:1034] running on GCE node
I20260925 02:27:35.013360 20390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:35.013581 20390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:35.014745 20390 hybrid_clock.cc:648] HybridClock initialized: now 1790303255014730 us; error 25 us; skew 500 ppm
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303255, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:35.022009 20390 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:35.022413 20390 webserver.cc:460] Webserver started at http://127.11.220.254:33663/ using document root <none> and password file <none>
I20260925 02:27:35.022629 20390 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:35.022691 20390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:35.022825 20390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:35.023726 20390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "0a24072b5baa442a9bb34daac4d9144a"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "395a9dd5ac96cb7a6a93f30a2b455998"
server_key_iv: "c23583134cb6a306ea0a994eb73d4101"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.024041 20390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "0a24072b5baa442a9bb34daac4d9144a"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "395a9dd5ac96cb7a6a93f30a2b455998"
server_key_iv: "c23583134cb6a306ea0a994eb73d4101"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.025359 20390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:27:35.026054 20405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.026223 20390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:35.026324 20390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "0a24072b5baa442a9bb34daac4d9144a"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "395a9dd5ac96cb7a6a93f30a2b455998"
server_key_iv: "c23583134cb6a306ea0a994eb73d4101"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.026396 20390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:35.037299 20390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:35.039251 20390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:35.039389 20390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:35.043035 20390 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:40047
I20260925 02:27:35.043062 20458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:40047 every 8 connection(s)
I20260925 02:27:35.043347 20390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:35.044068 20459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:35.044874 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 20390
I20260925 02:27:35.044986 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:35.045171 12147 external_mini_cluster.cc:1428] Setting key 1370b7ff86bce15040b9d920016f73b2
I20260925 02:27:35.047070 20459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a: Bootstrap starting.
I20260925 02:27:35.047627 20459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:35.047873 20459 log.cc:824] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:35.048529 20459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a: No bootstrap required, opened a new log
I20260925 02:27:35.050521 20459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } }
I20260925 02:27:35.050679 20459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:35.050711 20459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a24072b5baa442a9bb34daac4d9144a, State: Initialized, Role: FOLLOWER
I20260925 02:27:35.050823 20459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } }
I20260925 02:27:35.050902 20459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:35.050930 20459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:35.050971 20459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 0 FOLLOWER]: Advancing to term 1
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:35.051611 20459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } }
I20260925 02:27:35.051693 20459 leader_election.cc:302] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a24072b5baa442a9bb34daac4d9144a; no voters:
I20260925 02:27:35.051885 20459 leader_election.cc:288] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:35.051913 20464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:35.052021 20464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 1 LEADER]: Becoming Leader. State: Replica: 0a24072b5baa442a9bb34daac4d9144a, State: Running, Role: LEADER
I20260925 02:27:35.052105 20459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:35.052112 20464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } }
I20260925 02:27:35.052531 20464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a24072b5baa442a9bb34daac4d9144a. Latest consensus state: current_term: 1 leader_uuid: "0a24072b5baa442a9bb34daac4d9144a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } } }
I20260925 02:27:35.052544 20467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a24072b5baa442a9bb34daac4d9144a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } } }
I20260925 02:27:35.052686 20464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:35.052695 20467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:35.052918 20473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:35.053436 20473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:35.055080 20473 catalog_manager.cc:1347] Generated new cluster ID: 5303bb4b797f4a84908bfa9a8d3f5f01
I20260925 02:27:35.055090 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.045653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33522 (local address 127.11.220.254:40047)
0925 02:27:35.045805 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:35.045818 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:35.045840 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:27:35.046205 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:35.046211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:35.046250 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:35.046347 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:35.047067 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.047905 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.048691 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.049227 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.049340 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.051546 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:35.051549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:35.051549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:35.051558 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:35.053317 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:35.053707 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:35.053709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:35.053709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:35.053804 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:35.053965 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:35.053966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:35.053966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:35.054213 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:35.054704 (+ 491us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:35.054782 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 02:27:35.054916 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260925 02:27:35.055181 20473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:35.064715 20473 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:35.065183 20473 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:35.069924 20473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a: Generated new TSK 0
I20260925 02:27:35.070070 20473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:35.096568 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40047
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:35.191092 20486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:35.191290 20486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:35.191323 20486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:35.192382 20486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:35.192447 20486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:35.192521 20486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:35.193857 20486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:35.194104 20486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:35.194307 20486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:35.198698 20494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.198714 20492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:35.198925 20486 server_base.cc:1034] running on GCE node
W20260925 02:27:35.199018 20491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:35.199224 20486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:35.199461 20486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:35.200608 20486 hybrid_clock.cc:648] HybridClock initialized: now 1790303255200596 us; error 27 us; skew 500 ppm
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303255, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:35.202885 20486 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:35.203302 20486 webserver.cc:460] Webserver started at http://127.11.220.193:39421/ using document root <none> and password file <none>
I20260925 02:27:35.203523 20486 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:35.203584 20486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:35.203737 20486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:35.204780 20486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "6bfd97699dcf405fa319584c29db9312"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "7021aa4b273c1e0d26aacba68ebe3e32"
server_key_iv: "72e1760b94c586deda1a7b630f682787"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.205119 20486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "6bfd97699dcf405fa319584c29db9312"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "7021aa4b273c1e0d26aacba68ebe3e32"
server_key_iv: "72e1760b94c586deda1a7b630f682787"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.206300 20486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:35.206926 20501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.207089 20486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:35.207176 20486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "6bfd97699dcf405fa319584c29db9312"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "7021aa4b273c1e0d26aacba68ebe3e32"
server_key_iv: "72e1760b94c586deda1a7b630f682787"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.207250 20486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:35.219854 20486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:35.222002 20486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:35.222149 20486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:35.222349 20486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:35.222621 20486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:35.222671 20486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.222699 20486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:35.222723 20486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.228204 20486 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:46191
I20260925 02:27:35.228386 20617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:46191 every 8 connection(s)
I20260925 02:27:35.228569 20486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:35.230993 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 20486
I20260925 02:27:35.231102 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:35.231285 12147 external_mini_cluster.cc:1428] Setting key 5a0b80610d1634270c80e18ca4941418
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303255, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:35.237785 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.229207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:58437 (local address 127.11.220.254:40047)
0925 02:27:35.229270 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:35.229271 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:35.229445 (+ 174us) server_negotiation.cc:407] Connection header received
0925 02:27:35.229848 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:35.229849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:35.229858 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:35.229909 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:35.230863 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.231566 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.232652 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.232969 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.233034 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.235108 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:35.235109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:35.235110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:35.235117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:35.236501 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:35.236861 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:35.236862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:35.236862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:35.236919 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:35.237171 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:35.237172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:35.237172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:35.237278 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:35.237347 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:35.237611 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 02:27:35.237701 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:27:35.237924 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.229197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:58437)
0925 02:27:35.229329 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:35.229364 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:35.229775 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:35.229954 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:35.229957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:35.230592 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:35.230766 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.230771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.231650 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.231651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.232559 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.232562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.233053 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.233054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.233195 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.233622 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:35.233632 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:35.235008 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:35.236575 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:35.236582 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:35.236584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:35.236788 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:35.236993 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:35.236994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:35.236994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:35.237110 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:35.237339 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:35.237345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:35.237484 (+ 139us) client_negotiation.cc:769] Sending connection context
0925 02:27:35.237583 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 02:27:35.237722 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":72,"threads_started":1}
I20260925 02:27:35.238379 20618 heartbeater.cc:347] Connected to a master server at 127.11.220.254:40047
I20260925 02:27:35.238476 20618 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:35.238632 20618 heartbeater.cc:511] Master 127.11.220.254:40047 requested a full tablet report, sending...
I20260925 02:27:35.239147 20422 ts_manager.cc:194] Registered new tserver with Master: 6bfd97699dcf405fa319584c29db9312 (127.11.220.193:46191)
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
I20260925 02:27:35.239693 20422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:58437
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:35.254623 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40047
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:35.328212 20623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:35.328413 20623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:35.328459 20623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:35.329524 20623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:35.329583 20623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:35.329619 20623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:35.330996 20623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:35.331226 20623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:35.331426 20623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:35.335738 20629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.335732 20628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.335736 20631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:35.335762 20623 server_base.cc:1034] running on GCE node
I20260925 02:27:35.336104 20623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:35.336323 20623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:35.337474 20623 hybrid_clock.cc:648] HybridClock initialized: now 1790303255337458 us; error 28 us; skew 500 ppm
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303255, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:35.339577 20623 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:35.339993 20623 webserver.cc:460] Webserver started at http://127.11.220.194:42583/ using document root <none> and password file <none>
I20260925 02:27:35.340193 20623 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:35.340255 20623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:35.340380 20623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:35.341276 20623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "eda3525c5c9e4429adb4fb14ab096bcc"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "7cdcb5ddfa9475c6c4dd851766a9c246"
server_key_iv: "c6ca6448d4f9afc7961305f288eaf081"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.341594 20623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "eda3525c5c9e4429adb4fb14ab096bcc"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "7cdcb5ddfa9475c6c4dd851766a9c246"
server_key_iv: "c6ca6448d4f9afc7961305f288eaf081"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.342715 20623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:35.343324 20638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.343493 20623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:35.343578 20623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "eda3525c5c9e4429adb4fb14ab096bcc"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "7cdcb5ddfa9475c6c4dd851766a9c246"
server_key_iv: "c6ca6448d4f9afc7961305f288eaf081"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.343660 20623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:35.350358 20623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:35.352231 20623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:35.352361 20623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:35.352612 20623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:35.352901 20623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:35.352979 20623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.353016 20623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:35.353042 20623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.360067 20623 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:41281
I20260925 02:27:35.360162 20754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:41281 every 8 connection(s)
I20260925 02:27:35.360468 20623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303255, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:35.368726 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 20623
I20260925 02:27:35.368834 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:35.369006 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.361185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:59225 (local address 127.11.220.254:40047)
0925 02:27:35.361255 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:35.361258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:35.361442 (+ 184us) server_negotiation.cc:407] Connection header received
0925 02:27:35.361970 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:35.361971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:35.361983 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:35.362039 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:35.363130 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.363615 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.364555 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.364756 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.364826 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.366616 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:35.366617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:35.366617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:35.366624 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:35.367683 (+ 1059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:35.368069 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:35.368070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:35.368070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:35.368130 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:35.368363 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:35.368364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:35.368364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:35.368527 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:35.368612 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:35.368851 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 02:27:35.368923 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:27:35.369017 12147 external_mini_cluster.cc:1428] Setting key 56f69ff7d0be5feceef7af3d4c83e86c
I20260925 02:27:35.369189 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.361147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:59225)
0925 02:27:35.361315 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:35.361356 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:35.361863 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:35.362107 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:35.362110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:35.362831 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:35.363017 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.363022 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.363703 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.363704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.364490 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.364492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.364833 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.364834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.364953 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.365329 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:35.365336 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:35.366519 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:35.367759 (+ 1240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:35.367766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:35.367768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:35.367998 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:35.368193 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:35.368194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:35.368194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:35.368293 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:35.368595 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:35.368602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:35.368729 (+ 127us) client_negotiation.cc:769] Sending connection context
0925 02:27:35.368830 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 02:27:35.368970 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260925 02:27:35.369728 20755 heartbeater.cc:347] Connected to a master server at 127.11.220.254:40047
I20260925 02:27:35.369838 20755 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:35.370011 20755 heartbeater.cc:511] Master 127.11.220.254:40047 requested a full tablet report, sending...
I20260925 02:27:35.370388 20421 ts_manager.cc:194] Registered new tserver with Master: eda3525c5c9e4429adb4fb14ab096bcc (127.11.220.194:41281)
I20260925 02:27:35.370733 20421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:59225
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:35.388933 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40047
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:35.461777 20760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:35.461931 20760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:35.461951 20760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:35.462966 20760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:35.463017 20760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:35.463063 20760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:35.464335 20760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:35.464572 20760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:35.464808 20760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:35.469091 20766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.469097 20765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:35.469151 20760 server_base.cc:1034] running on GCE node
W20260925 02:27:35.469148 20768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:35.469549 20760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:35.469772 20760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:35.470932 20760 hybrid_clock.cc:648] HybridClock initialized: now 1790303255470919 us; error 24 us; skew 500 ppm
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303255, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:35.473183 20760 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:35.473630 20760 webserver.cc:460] Webserver started at http://127.11.220.195:33155/ using document root <none> and password file <none>
I20260925 02:27:35.473841 20760 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:35.473902 20760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:35.474020 20760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:35.474925 20760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "0a97ed2ee8544809b3b19da578635fac"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "ff709d3c75f9e1d0b2d029178cf5c6ee"
server_key_iv: "ab2591daf6c0c5070034feb17458e7da"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.475251 20760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "0a97ed2ee8544809b3b19da578635fac"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "ff709d3c75f9e1d0b2d029178cf5c6ee"
server_key_iv: "ab2591daf6c0c5070034feb17458e7da"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.476425 20760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:27:35.477167 20775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.477334 20760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:35.477419 20760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "0a97ed2ee8544809b3b19da578635fac"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "ff709d3c75f9e1d0b2d029178cf5c6ee"
server_key_iv: "ab2591daf6c0c5070034feb17458e7da"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.477496 20760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:35.482788 20760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:35.484267 20760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:35.484407 20760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:35.484613 20760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:35.484874 20760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:35.484921 20760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.484948 20760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:35.484975 20760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.490239 20760 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:39879
I20260925 02:27:35.490268 20890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:39879 every 8 connection(s)
I20260925 02:27:35.490523 20760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:35.492254 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 20760
I20260925 02:27:35.492388 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:35.492568 12147 external_mini_cluster.cc:1428] Setting key d55ab7165fd3cbfa98fa033da6dfecc4
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303255, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:35.498227 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.491111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:40937 (local address 127.11.220.254:40047)
0925 02:27:35.491178 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:35.491180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:35.491337 (+ 157us) server_negotiation.cc:407] Connection header received
0925 02:27:35.491718 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:35.491719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:35.491728 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:35.491772 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:35.492642 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.493322 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.494192 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.494358 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.494405 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.496144 (+ 1739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:35.496145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:35.496145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:35.496151 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:35.497265 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:35.497577 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:35.497578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:35.497578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:35.497638 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:35.497820 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:35.497820 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:35.497820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:35.497899 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:35.497940 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:35.498124 (+ 184us) server_negotiation.cc:299] Negotiation successful
0925 02:27:35.498173 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:27:35.498410 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.491057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:40937)
0925 02:27:35.491239 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:35.491268 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:35.491646 (+ 378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:35.491815 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:35.491817 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:35.492301 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:35.492550 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.492554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.493412 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.493413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.494126 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.494129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.494424 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.494424 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.494527 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.494883 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:35.494890 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:35.496076 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:35.497328 (+ 1252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:35.497334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:35.497335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:35.497520 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:35.497693 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:35.497693 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:35.497694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:35.497775 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:35.497945 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:35.497948 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:35.498035 (+ 87us) client_negotiation.cc:769] Sending connection context
0925 02:27:35.498099 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:27:35.498194 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260925 02:27:35.498842 20891 heartbeater.cc:347] Connected to a master server at 127.11.220.254:40047
I20260925 02:27:35.498930 20891 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:35.499075 20891 heartbeater.cc:511] Master 127.11.220.254:40047 requested a full tablet report, sending...
I20260925 02:27:35.499379 20421 ts_manager.cc:194] Registered new tserver with Master: 0a97ed2ee8544809b3b19da578635fac (127.11.220.195:39879)
I20260925 02:27:35.499698 20421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:40937
I20260925 02:27:35.504143 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab.
Entry for principal oryx/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab.
WARNING: no policy specified for oryx/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab.
Entry for principal oryx/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab.
WARNING: no policy specified for oryx/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab.
Entry for principal oryx/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab.
I20260925 02:27:35.562451 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 20486
I20260925 02:27:35.567308 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 20623
I20260925 02:27:35.571560 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 20760
I20260925 02:27:35.575780 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 20390
I20260925 02:27:35.580431 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:40047
--webserver_interface=127.11.220.254
--webserver_port=33663
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:40047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:35.652794 20900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:35.652985 20900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:35.653019 20900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:35.654050 20900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:35.654109 20900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:35.654129 20900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:35.654150 20900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:35.654170 20900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:35.655603 20900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:40047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:40047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=33663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:35.655879 20900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:35.656093 20900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:35.660530 20906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.660559 20905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.660555 20908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:35.660545 20900 server_base.cc:1034] running on GCE node
I20260925 02:27:35.660907 20900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:35.661139 20900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:35.662266 20900 hybrid_clock.cc:648] HybridClock initialized: now 1790303255662254 us; error 25 us; skew 500 ppm
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303255, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:35.664541 20900 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:35.664896 20900 webserver.cc:460] Webserver started at http://127.11.220.254:33663/ using document root <none> and password file <none>
I20260925 02:27:35.665073 20900 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:35.665122 20900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:35.666296 20900 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:35.666826 20915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.666988 20900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:35.667044 20900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "0a24072b5baa442a9bb34daac4d9144a"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "395a9dd5ac96cb7a6a93f30a2b455998"
server_key_iv: "c23583134cb6a306ea0a994eb73d4101"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.667328 20900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:35.700076 20900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:35.701733 20900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:35.701870 20900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:35.705438 20900 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:40047
I20260925 02:27:35.705482 20967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:40047 every 8 connection(s)
I20260925 02:27:35.705837 20900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:35.707111 20968 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:27:35.707715 20968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a: Bootstrap starting.
I20260925 02:27:35.714880 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 20900
I20260925 02:27:35.715185 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:46191
--local_ip_for_outbound_sockets=127.11.220.193
--tserver_master_addrs=127.11.220.254:40047
--webserver_port=39421
--webserver_interface=127.11.220.193
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.220.193
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:27:35.716234 20968 log.cc:824] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:35.717826 20968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:27:35.718091 20968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a: Bootstrap complete.
I20260925 02:27:35.720286 20968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } }
I20260925 02:27:35.720568 20968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a24072b5baa442a9bb34daac4d9144a, State: Initialized, Role: FOLLOWER
I20260925 02:27:35.720775 20968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } }
I20260925 02:27:35.720849 20968 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:35.720899 20968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:35.720944 20968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:27:35.721871 20968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } }
I20260925 02:27:35.721980 20968 leader_election.cc:302] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a24072b5baa442a9bb34daac4d9144a; no voters:
I20260925 02:27:35.722137 20968 leader_election.cc:288] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 02:27:35.722184 20972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 2 FOLLOWER]: Leader election won for term 2
I20260925 02:27:35.722297 20972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [term 2 LEADER]: Becoming Leader. State: Replica: 0a24072b5baa442a9bb34daac4d9144a, State: Running, Role: LEADER
I20260925 02:27:35.722381 20968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:35.722373 20972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } }
I20260925 02:27:35.722635 20973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0a24072b5baa442a9bb34daac4d9144a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } } }
I20260925 02:27:35.722720 20973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:35.723012 20977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:35.723021 20974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a24072b5baa442a9bb34daac4d9144a. Latest consensus state: current_term: 2 leader_uuid: "0a24072b5baa442a9bb34daac4d9144a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a24072b5baa442a9bb34daac4d9144a" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40047 } } }
I20260925 02:27:35.723371 20974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:35.723515 20977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:35.723634 20977 catalog_manager.cc:1259] Loaded cluster ID: 5303bb4b797f4a84908bfa9a8d3f5f01
I20260925 02:27:35.723671 20977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:35.724784 20977 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:35.724984 20977 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0a24072b5baa442a9bb34daac4d9144a: Loaded TSK: 0
I20260925 02:27:35.725198 20977 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:27:35.791489 20970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:35.791680 20970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:35.791714 20970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:35.792827 20970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:35.792881 20970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:35.792919 20970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:35.794214 20970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.193.keytab
--principal=oryx/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:46191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=39421
--tserver_master_addrs=127.11.220.254:40047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:35.794433 20970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:35.794624 20970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:35.799883 20994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.799892 20995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.799901 20997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:35.800279 20970 server_base.cc:1034] running on GCE node
I20260925 02:27:35.800496 20970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:35.800731 20970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:35.801887 20970 hybrid_clock.cc:648] HybridClock initialized: now 1790303255801876 us; error 28 us; skew 500 ppm
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:35.804165 20970 init.cc:377] Logged in from keytab as oryx/127.11.220.193@KRBTEST.COM (short username oryx)
I20260925 02:27:35.804618 20970 webserver.cc:460] Webserver started at http://127.11.220.193:39421/ using document root <none> and password file <none>
I20260925 02:27:35.804836 20970 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:35.804912 20970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:35.806125 20970 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:35.806725 21004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.806891 20970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.806978 20970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "6bfd97699dcf405fa319584c29db9312"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "7021aa4b273c1e0d26aacba68ebe3e32"
server_key_iv: "72e1760b94c586deda1a7b630f682787"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.807291 20970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:35.813772 20970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:35.815675 20970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:35.815814 20970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:35.816059 20970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:35.816335 20970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:35.816388 20970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.816425 20970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:35.816490 20970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.822111 20970 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:46191
I20260925 02:27:35.822202 21121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:46191 every 8 connection(s)
I20260925 02:27:35.822556 20970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:35.829758 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 20970
I20260925 02:27:35.830096 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:41281
--local_ip_for_outbound_sockets=127.11.220.194
--tserver_master_addrs=127.11.220.254:40047
--webserver_port=42583
--webserver_interface=127.11.220.194
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.220.194
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:35.832000 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.823236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:35341)
0925 02:27:35.823402 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:35.823442 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:35.824068 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:35.824902 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:35.824906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:35.825622 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:35.825871 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.825876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.826954 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.826956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.828194 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.828197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.828745 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.828746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.828884 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.829331 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:35.829340 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:35.831815 (+ 2475us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
W20260925 02:27:35.832198 21125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:35.823370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:35341 (local address 127.11.220.254:40047)
0925 02:27:35.824552 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:35.824560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:35.824585 (+ 25us) server_negotiation.cc:407] Connection header received
0925 02:27:35.824676 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:35.824681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:35.824718 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:35.824815 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:35.825976 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.826852 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.828274 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.828662 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.828744 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.831922 (+ 3178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:35341: BlockingRecv error: recv got EOF from 127.11.220.193:35341 (error 108)
Metrics: {"server-negotiator.queue_time_us":1141,"thread_start_us":74,"threads_started":1}
W20260925 02:27:35.832249 21122 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:40047 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.220.254:40047: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:35.836627 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.832627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:43095)
0925 02:27:35.832674 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:35.832685 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:35.832722 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:35.832898 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:35.832899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:35.833022 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:35.833061 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.833062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.833669 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.833670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.834115 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.834117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.834497 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.834497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.834563 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.834951 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:35.834956 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:35.836555 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 02:27:35.836670 21125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:35.832729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:43095 (local address 127.11.220.254:40047)
0925 02:27:35.832788 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:35.832790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:35.832796 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:27:35.832815 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:35.832816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:35.832828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:35.832895 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:35.833147 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.833599 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.834261 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.834418 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.834494 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.836617 (+ 2123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:43095: BlockingRecv error: recv got EOF from 127.11.220.193:43095 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:35.840922 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.837029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:40575)
0925 02:27:35.837109 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:35.837118 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:35.837156 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:35.837276 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:35.837276 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:35.837389 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:35.837410 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.837411 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.837983 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.837984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.838473 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.838474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.838824 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.838825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.838876 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.839298 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:35.839303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:35.840858 (+ 1555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260925 02:27:35.840965 21125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:35.837078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:40575 (local address 127.11.220.254:40047)
0925 02:27:35.837113 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:35.837114 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:35.837145 (+ 31us) server_negotiation.cc:407] Connection header received
0925 02:27:35.837215 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:35.837216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:35.837224 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:35.837272 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:35.837483 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.837909 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.838566 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.838753 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.838825 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.840914 (+ 2089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:40575: BlockingRecv error: recv got EOF from 127.11.220.193:40575 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260925 02:27:35.841221 21122 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:27:35.912634 21126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:35.912806 21126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:35.912828 21126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:35.913880 21126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:35.913911 21126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:35.913936 21126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:35.915225 21126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.194.keytab
--principal=oryx/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:41281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=42583
--tserver_master_addrs=127.11.220.254:40047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:35.915417 21126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:35.915573 21126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:35.920233 21134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.920212 21131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:35.920219 21132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:35.920343 21126 server_base.cc:1034] running on GCE node
I20260925 02:27:35.920688 21126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:35.920917 21126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:35.922082 21126 hybrid_clock.cc:648] HybridClock initialized: now 1790303255922069 us; error 24 us; skew 500 ppm
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303255, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:35.924629 21126 init.cc:377] Logged in from keytab as oryx/127.11.220.194@KRBTEST.COM (short username oryx)
I20260925 02:27:35.925014 21126 webserver.cc:460] Webserver started at http://127.11.220.194:42583/ using document root <none> and password file <none>
I20260925 02:27:35.925235 21126 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:35.925287 21126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:35.926559 21126 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:35.927104 21141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.927258 21126 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:35.927351 21126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "eda3525c5c9e4429adb4fb14ab096bcc"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "7cdcb5ddfa9475c6c4dd851766a9c246"
server_key_iv: "c6ca6448d4f9afc7961305f288eaf081"
server_key_version: "encryptionkey@0"
I20260925 02:27:35.927647 21126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:35.949800 21126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:35.951859 21126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:35.952032 21126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:35.952207 21126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:35.952482 21126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:35.952523 21126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.952548 21126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:35.952571 21126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:35.958462 21126 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:41281
I20260925 02:27:35.958510 21258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:41281 every 8 connection(s)
I20260925 02:27:35.958814 21126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:35.965736 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 21126
W20260925 02:27:35.965930 21125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:35.959478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:52859 (local address 127.11.220.254:40047)
0925 02:27:35.959539 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:35.959541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:35.959755 (+ 214us) server_negotiation.cc:407] Connection header received
0925 02:27:35.960261 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:35.960263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:35.960278 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:35.960332 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:35.961223 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.961705 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.962681 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.962895 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.962975 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.965885 (+ 2910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:52859: BlockingRecv error: recv got EOF from 127.11.220.194:52859 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:27:35.965982 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.959481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:52859)
0925 02:27:35.959634 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:35.959672 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:35.960157 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:35.960396 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:35.960406 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:35.960943 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:35.961118 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.961123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.961803 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.961804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.962579 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.962581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.962986 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.962987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.963104 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.963482 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:35.963490 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:35.965780 (+ 2290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260925 02:27:35.966032 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:39879
--local_ip_for_outbound_sockets=127.11.220.195
--tserver_master_addrs=127.11.220.254:40047
--webserver_port=33155
--webserver_interface=127.11.220.195
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.220.195
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:35.966233 21259 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:40047 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.220.254:40047: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:35.970568 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.966437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:55501)
0925 02:27:35.966485 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:35.966495 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:35.966549 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:35.966684 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:35.966685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:35.966796 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:35.966834 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.966835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.967610 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.967611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.968069 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.968070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.968410 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.968411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.968497 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.968898 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:35.968903 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:35.970498 (+ 1595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 02:27:35.970623 21125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:35.966497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:55501 (local address 127.11.220.254:40047)
0925 02:27:35.966553 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:35.966555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:35.966563 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:35.966613 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:35.966614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:35.966622 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:35.966681 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:35.966923 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.967525 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.968149 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.968348 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.968397 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.970560 (+ 2163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:55501: BlockingRecv error: recv got EOF from 127.11.220.194:55501 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:35 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:35.974882 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:35.970820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:37759)
0925 02:27:35.970902 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:35.970911 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:35.970956 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:35.971088 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:35.971089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:35.971206 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:35.971227 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.971228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.971804 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.971805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.972270 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:35.972272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.972679 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.972680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:35.972733 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.973118 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:35.973123 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:35.974801 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":71}
W20260925 02:27:35.974927 21125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:35.970850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:37759 (local address 127.11.220.254:40047)
0925 02:27:35.970881 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:35.970883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:35.970957 (+ 74us) server_negotiation.cc:407] Connection header received
0925 02:27:35.971009 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:35.971010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:35.971018 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:35.971085 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:35.971305 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.971732 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.972354 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:35.972596 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:35.972677 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:35.974874 (+ 2197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:37759: BlockingRecv error: recv got EOF from 127.11.220.194:37759 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 02:27:35.975153 21259 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:27:36.044351 21262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:36.044605 21262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:36.044631 21262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:36.045753 21262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:36.045790 21262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:36.045815 21262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:36.047143 21262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/krb5kdc/oryx_127.11.220.195.keytab
--principal=oryx/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:39879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=33155
--tserver_master_addrs=127.11.220.254:40047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:36.047351 21262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:36.047554 21262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:36.052204 21270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:36.052209 21267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:36.052212 21268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:36.052345 21262 server_base.cc:1034] running on GCE node
I20260925 02:27:36.052634 21262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:36.052904 21262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:36.054069 21262 hybrid_clock.cc:648] HybridClock initialized: now 1790303256054055 us; error 29 us; skew 500 ppm
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303256, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:36.056514 21262 init.cc:377] Logged in from keytab as oryx/127.11.220.195@KRBTEST.COM (short username oryx)
I20260925 02:27:36.056856 21262 webserver.cc:460] Webserver started at http://127.11.220.195:33155/ using document root <none> and password file <none>
I20260925 02:27:36.057034 21262 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:36.057089 21262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:36.058367 21262 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:36.058868 21277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:36.059054 21262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:36.059147 21262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "0a97ed2ee8544809b3b19da578635fac"
format_stamp: "Formatted at 2026-09-25 02:27:35 on dist-test-slave-dn4n"
server_key: "ff709d3c75f9e1d0b2d029178cf5c6ee"
server_key_iv: "ab2591daf6c0c5070034feb17458e7da"
server_key_version: "encryptionkey@0"
I20260925 02:27:36.059419 21262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:36.074543 21262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:36.076189 21262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:36.076292 21262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:36.076526 21262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:36.076761 21262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:36.076788 21262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:36.076807 21262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:36.076818 21262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:36.082022 21262 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:39879
I20260925 02:27:36.082159 21393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:39879 every 8 connection(s)
I20260925 02:27:36.082474 21262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 02:27:36.089282 21125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:36.083252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:42079 (local address 127.11.220.254:40047)
0925 02:27:36.083326 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:36.083328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:36.083542 (+ 214us) server_negotiation.cc:407] Connection header received
0925 02:27:36.083970 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:36.083970 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:36.083979 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:36.084030 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:36.084936 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.085396 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.086384 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.086551 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.086599 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.089238 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:42079: BlockingRecv error: recv got EOF from 127.11.220.195:42079 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 02:27:36.089289 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:36.083246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:42079)
0925 02:27:36.083428 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:36.083464 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:36.083896 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:36.084079 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:36.084082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:36.084672 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:36.084855 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:36.084860 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.085489 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.085489 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:36.086294 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:36.086296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.086619 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.086620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:36.086725 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.087194 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:36.087202 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:36.089162 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":116,"threads_started":1}
W20260925 02:27:36.089558 21394 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:40047 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.220.254:40047: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
I20260925 02:27:36.091032 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 21262
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:36.094061 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:36.089775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:42593)
0925 02:27:36.089820 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:36.089830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:36.089865 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:36.090067 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:36.090068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:36.090174 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:36.090213 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:36.090214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.090731 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.090732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:36.091349 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:36.091351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.091737 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.091738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:36.091806 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.092172 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:36.092177 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:36.093989 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 02:27:36.094108 21125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:36.089934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:42593 (local address 127.11.220.254:40047)
0925 02:27:36.089967 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:36.089969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:36.089978 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:36.089998 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:36.089999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:36.090005 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:36.090043 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:36.090273 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.090676 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.091446 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.091651 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.091734 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.094052 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:42593: BlockingRecv error: recv got EOF from 127.11.220.195:42593 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:36.098062 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:36.094330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:40623)
0925 02:27:36.094365 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:36.094373 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:36.094406 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:36.094519 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:36.094520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:36.094639 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:36.094659 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:36.094660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.095218 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.095219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:36.095854 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:36.095857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.096192 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.096193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:36.096243 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.096630 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:36.096651 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:36.098028 (+ 1377us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260925 02:27:36.098197 21394 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:27:36.098297 21125 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:36.094375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:40623 (local address 127.11.220.254:40047)
0925 02:27:36.094407 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:36.094409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:36.094416 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:36.094456 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:36.094456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:36.094464 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:36.094515 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:36.094728 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.095144 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.095923 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.096098 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.096190 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.098111 (+ 1921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:40623: BlockingRecv error: recv got EOF from 127.11.220.195:40623 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:36.099022 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:36.091399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.11.220.254:40047)
0925 02:27:36.091499 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:36.091500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:36.091613 (+ 113us) server_negotiation.cc:407] Connection header received
0925 02:27:36.091685 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:36.091685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:36.091696 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:36.091743 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:36.092314 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.093072 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.094055 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.094355 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.094416 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.095540 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:36.095544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:36.095545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:36.095568 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:36.097527 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:36.098131 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:36.098134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:36.098134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:36.098272 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:36.098449 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:36.098450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:36.098450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:36.098643 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:36.098747 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:36.098827 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 02:27:36.098899 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":59,"threads_started":1}
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:36.845959 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:36.841568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:58813)
0925 02:27:36.841693 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:36.841701 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:36.841741 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:36.841879 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:36.841880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:36.842015 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:36.842059 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:36.842060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.842877 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.842878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:36.843366 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:36.843368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.843652 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.843653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:36.843689 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.844064 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:36.844068 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:36.845896 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
W20260925 02:27:36.846043 21401 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:36.841655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:58813 (local address 127.11.220.254:40047)
0925 02:27:36.841774 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:36.841776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:36.841782 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:27:36.841801 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:36.841801 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:36.841809 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:36.841862 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:36.842126 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.842814 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.843429 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.843596 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.843640 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.845976 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:58813: BlockingRecv error: recv got EOF from 127.11.220.193:58813 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:36 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:36.979600 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:36.975499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:34471)
0925 02:27:36.975625 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:36.975635 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:36.975703 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:36.975834 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:36.975834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:36.975990 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:36.976035 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:36.976037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.976671 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.976671 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:36.977150 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:36.977151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.977474 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.977475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:36.977511 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.977881 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:36.977885 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:36.979543 (+ 1658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
W20260925 02:27:36.979722 21401 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:36.975568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:34471 (local address 127.11.220.254:40047)
0925 02:27:36.975616 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:36.975618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:36.975682 (+ 64us) server_negotiation.cc:407] Connection header received
0925 02:27:36.975752 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:36.975752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:36.975761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:36.975817 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:36.976090 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.976611 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.977213 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:36.977406 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:36.977464 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:36.979645 (+ 2181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:34471: BlockingRecv error: recv got EOF from 127.11.220.194:34471 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 02:27:37 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:37 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:37.102486 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:37.098521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:55969)
0925 02:27:37.098629 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:37.098640 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:37.098698 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:37.098811 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:37.098812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:37.098947 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:37.098990 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:37.098991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.099555 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.099555 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:37.100043 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:37.100045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.100325 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.100325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:37.100380 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:37.100775 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:37.100779 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:37.102425 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
W20260925 02:27:37.102574 21401 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:37.098571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:55969 (local address 127.11.220.254:40047)
0925 02:27:37.098626 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:37.098629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:37.098686 (+ 57us) server_negotiation.cc:407] Connection header received
0925 02:27:37.098757 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:37.098757 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:37.098766 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:37.098806 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:37.099045 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.099494 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.100107 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.100269 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.100313 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:37.102513 (+ 2200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:55969: BlockingRecv error: recv got EOF from 127.11.220.195:55969 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 25 02:27:37 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:37 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:37.850734 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:37.846536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:40281)
0925 02:27:37.846671 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:37.846680 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:37.846722 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:37.846839 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:37.846839 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:37.846995 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:37.847054 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:37.847055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.847595 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.847595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:37.848039 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:37.848040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.848353 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.848353 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:37.848388 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:37.848892 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:37.848897 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:37.850684 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
W20260925 02:27:37.850878 21405 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:37.846614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:40281 (local address 127.11.220.254:40047)
0925 02:27:37.846723 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:37.846725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:37.846751 (+ 26us) server_negotiation.cc:407] Connection header received
0925 02:27:37.846769 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:37.846770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:37.846779 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:37.846823 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:37.847117 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.847538 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.848110 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.848293 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.848340 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:37.850799 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:40281: BlockingRecv error: recv got EOF from 127.11.220.193:40281 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
Sep 25 02:27:37 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:37 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:37.983942 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:37.980088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:42867)
0925 02:27:37.980237 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:37.980246 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:37.980295 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:37.980400 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:37.980401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:37.980559 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:37.980606 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:37.980608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.981184 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.981185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:37.981643 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:37.981643 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.981920 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.981921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:37.981958 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:37.982330 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:37.982334 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:37.983892 (+ 1558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
W20260925 02:27:37.984068 21405 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:37.980163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:42867 (local address 127.11.220.254:40047)
0925 02:27:37.980234 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:37.980236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:37.980299 (+ 63us) server_negotiation.cc:407] Connection header received
0925 02:27:37.980332 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:37.980332 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:37.980341 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:37.980387 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:37.980661 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.981109 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.981704 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:37.981865 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:37.981908 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:37.983994 (+ 2086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:42867: BlockingRecv error: recv got EOF from 127.11.220.194:42867 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 02:27:38 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:38 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:38.107249 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:38.103127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:53471)
0925 02:27:38.103254 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:38.103280 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:38.103338 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:38.103493 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:38.103494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:38.103650 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:38.103697 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:38.103698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.104288 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.104289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:38.104800 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:38.104802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.105158 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.105160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:38.105226 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:38.105646 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:38.105650 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:38.107197 (+ 1547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":50,"threads_started":1}
W20260925 02:27:38.107384 21405 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:38.103240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:53471 (local address 127.11.220.254:40047)
0925 02:27:38.103310 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:38.103311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:38.103322 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:27:38.103409 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:38.103410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:38.103430 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:38.103488 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:38.103793 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.104216 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.104881 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.105074 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.105158 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:38.107284 (+ 2126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:53471: BlockingRecv error: recv got EOF from 127.11.220.195:53471 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 02:27:38 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:38 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:38.855595 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:38.851237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:36893)
0925 02:27:38.851380 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:38.851389 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:38.851441 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:38.851567 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:38.851568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:38.851745 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:38.851794 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:38.851795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.852551 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.852551 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:38.853062 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:38.853063 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.853399 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.853399 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:38.853438 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:38.854012 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:38.854017 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:38.855541 (+ 1524us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
W20260925 02:27:38.855710 21409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:38.851324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:36893 (local address 127.11.220.254:40047)
0925 02:27:38.851441 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:38.851443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:38.851478 (+ 35us) server_negotiation.cc:407] Connection header received
0925 02:27:38.851496 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:38.851497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:38.851507 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:38.851550 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:38.851887 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.852428 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.853131 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.853327 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.853388 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:38.855629 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:36893: BlockingRecv error: recv got EOF from 127.11.220.193:36893 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
Sep 25 02:27:38 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:38 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:38.988402 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:38.984497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:59987)
0925 02:27:38.984645 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:38.984656 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:38.984725 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:38.984877 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:38.984878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:38.985024 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:38.985083 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:38.985085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.985638 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.985638 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:38.986110 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:38.986111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.986397 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.986397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:38.986433 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:38.986822 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:38.986826 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:38.988352 (+ 1526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
W20260925 02:27:38.988548 21409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:38.984713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:59987 (local address 127.11.220.254:40047)
0925 02:27:38.984758 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:38.984760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:38.984768 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:38.984799 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:38.984799 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:38.984808 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:38.984854 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:38.985143 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.985581 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.986182 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:38.986342 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:38.986385 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:38.988480 (+ 2095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:59987: BlockingRecv error: recv got EOF from 127.11.220.194:59987 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 25 02:27:39 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:39 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:39.112381 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:39.107804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:54241)
0925 02:27:39.107929 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:39.107940 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:39.108000 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:39.108124 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:39.108125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:39.108276 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:39.108324 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:39.108325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.109177 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.109178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:39.109743 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:39.109745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.110020 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.110021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:39.110089 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:39.110531 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:39.110535 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:39.112292 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
W20260925 02:27:39.112429 21409 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:39.107869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:54241 (local address 127.11.220.254:40047)
0925 02:27:39.107931 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:39.107933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:39.108002 (+ 69us) server_negotiation.cc:407] Connection header received
0925 02:27:39.108042 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:39.108043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:39.108058 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:39.108107 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:39.108379 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.109091 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.109809 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.109963 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.110009 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:39.112364 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:54241: BlockingRecv error: recv got EOF from 127.11.220.195:54241 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 25 02:27:39 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:39 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:39.860191 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:39.856209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:56317)
0925 02:27:39.856349 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:39.856366 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:39.856411 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:39.856540 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:39.856541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:39.856688 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:39.856738 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:39.856739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.857305 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.857306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:39.857779 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:39.857780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.858064 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.858065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:39.858101 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:39.858486 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:39.858490 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:39.860113 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
W20260925 02:27:39.860261 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:39.856293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:56317 (local address 127.11.220.254:40047)
0925 02:27:39.856411 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:39.856413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:39.856428 (+ 15us) server_negotiation.cc:407] Connection header received
0925 02:27:39.856463 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:39.856463 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:39.856472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:39.856527 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:39.856798 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.857243 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.857842 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.858007 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.858050 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:39.860194 (+ 2144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:56317: BlockingRecv error: recv got EOF from 127.11.220.193:56317 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
Sep 25 02:27:39 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:39 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 02:27:39.994455 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:39.988999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:41087 (local address 127.11.220.254:40047)
0925 02:27:39.989067 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:39.989069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:39.989077 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:39.989205 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:39.989206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:39.989220 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:39.989288 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:39.989632 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.990111 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.991135 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.991461 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.991513 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:39.994379 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:41087: BlockingRecv error: recv got EOF from 127.11.220.194:41087 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:27:39.994659 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:39.988851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:41087)
0925 02:27:39.988979 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:39.988992 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:39.989108 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:39.989298 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:39.989299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:39.989500 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:39.989562 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:39.989565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.990203 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.990205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:39.991027 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:39.991032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:39.991625 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:39.991626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:39.991693 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:39.992172 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:39.992178 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:39.994570 (+ 2392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
Sep 25 02:27:40 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:40 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:40.121771 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:40.112797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:56361)
0925 02:27:40.112920 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:40.112928 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:40.112977 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:40.113091 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:40.113092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:40.113238 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:40.113284 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:40.113286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.113835 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.113836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:40.114307 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:40.114308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.114594 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.114594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:40.114632 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:40.120070 (+ 5438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:40.120075 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:40.121686 (+ 1611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
W20260925 02:27:40.121886 21413 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:40.112865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:56361 (local address 127.11.220.254:40047)
0925 02:27:40.112916 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:40.112918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:40.112975 (+ 57us) server_negotiation.cc:407] Connection header received
0925 02:27:40.113015 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:40.113015 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:40.113026 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:40.113074 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:40.113337 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.113779 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.114364 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.114519 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.114582 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:40.121810 (+ 7228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:56361: BlockingRecv error: recv got EOF from 127.11.220.195:56361 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 25 02:27:40 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:40 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:40.864962 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:40.860581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:41049)
0925 02:27:40.860716 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:40.860725 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:40.860772 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:40.860911 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:40.860912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:40.861059 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:40.861116 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:40.861117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.861794 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.861795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:40.862320 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:40.862323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.862686 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.862687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:40.862766 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:40.863164 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:40.863168 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:40.864873 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260925 02:27:40.865018 21417 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:40.860656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:41049 (local address 127.11.220.254:40047)
0925 02:27:40.860806 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:40.860808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:40.860815 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:40.860834 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:40.860835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:40.860845 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:40.860898 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:40.861217 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.861733 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.862408 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.862608 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.862678 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:40.864946 (+ 2268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:41049: BlockingRecv error: recv got EOF from 127.11.220.193:41049 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
Sep 25 02:27:40 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:40 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:40.999296 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:40.995172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:55357)
0925 02:27:40.995315 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:40.995329 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:40.995380 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:40.995513 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:40.995514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:40.995671 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:40.995718 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:40.995719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.996305 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.996305 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:40.996850 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:40.996852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.997171 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.997172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:40.997210 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:40.997694 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:40.997699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:40.999241 (+ 1542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
W20260925 02:27:40.999398 21417 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:40.995302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:55357 (local address 127.11.220.254:40047)
0925 02:27:40.995365 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:40.995367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:40.995375 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:40.995440 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:40.995441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:40.995451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:40.995491 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:40.995776 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.996246 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.996917 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:40.997094 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:40.997151 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:40.999328 (+ 2177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:55357: BlockingRecv error: recv got EOF from 127.11.220.194:55357 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 02:27:41 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:41 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:41.126570 21420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:41.122171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:60025)
0925 02:27:41.122332 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:41.122346 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:41.122415 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:41.122568 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:41.122568 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:41.122733 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:41.122777 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:41.122779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:41.123498 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:41.123499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:41.124024 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:41.124027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:41.124378 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:41.124379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:41.124437 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:41.124919 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:41.124923 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:41.126487 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
W20260925 02:27:41.126611 21417 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:41.122230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:60025 (local address 127.11.220.254:40047)
0925 02:27:41.122295 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:41.122297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:41.122396 (+ 99us) server_negotiation.cc:407] Connection header received
0925 02:27:41.122472 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:41.122473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:41.122495 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:41.122549 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:41.122844 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:41.123417 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:41.124113 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:41.124301 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:41.124376 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:41.126556 (+ 2180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:60025: BlockingRecv error: recv got EOF from 127.11.220.195:60025 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 25 02:27:41 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:41 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:41.869624 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:41.865403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:37147)
0925 02:27:41.865522 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:41.865532 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:41.865577 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:41.865753 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:41.865754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:41.865900 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:41.865948 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:41.865949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:41.866713 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:41.866714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:41.867252 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:41.867254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:41.867550 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:41.867551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:41.867588 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:41.867971 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:41.867975 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:41.869564 (+ 1589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
W20260925 02:27:41.869756 21422 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:41.865484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:37147 (local address 127.11.220.254:40047)
0925 02:27:41.865639 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:41.865641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:41.865649 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:41.865669 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:41.865669 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:41.865680 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:41.865740 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:41.866011 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:41.866629 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:41.867314 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:41.867489 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:41.867539 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:41.869668 (+ 2129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:37147: BlockingRecv error: recv got EOF from 127.11.220.193:37147 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
Sep 25 02:27:41 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:41 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:42.003808 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:41.999746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:53053)
0925 02:27:41.999853 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:41.999863 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:41.999915 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:42.000028 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:42.000029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:42.000182 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:42.000230 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:42.000231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.000863 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.000864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:42.001361 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:42.001362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.001665 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.001666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:42.001701 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:42.002149 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:42.002153 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:42.003739 (+ 1586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":52,"threads_started":1}
W20260925 02:27:42.003903 21422 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:41.999808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:53053 (local address 127.11.220.254:40047)
0925 02:27:41.999878 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:41.999880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:41.999910 (+ 30us) server_negotiation.cc:407] Connection header received
0925 02:27:41.999966 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:41.999967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:41.999978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:42.000013 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:42.000286 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.000795 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.001425 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.001608 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.001652 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:42.003837 (+ 2185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:53053: BlockingRecv error: recv got EOF from 127.11.220.194:53053 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 25 02:27:42 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:42 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:42.130856 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:42.126967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:39909)
0925 02:27:42.127072 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:42.127084 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:42.127141 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:42.127266 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:42.127266 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:42.127410 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:42.127456 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:42.127458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.128007 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.128008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:42.128520 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:42.128523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.128793 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.128794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:42.128832 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:42.129202 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:42.129206 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:42.130805 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":51,"threads_started":1}
W20260925 02:27:42.131062 21422 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:42.127036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:39909 (local address 127.11.220.254:40047)
0925 02:27:42.127097 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:42.127099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:42.127136 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:27:42.127192 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:42.127193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:42.127204 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:42.127249 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:42.127503 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.127948 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.128575 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.128735 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.128779 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:42.130877 (+ 2098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:39909: BlockingRecv error: recv got EOF from 127.11.220.195:39909 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 02:27:42 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:42 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:42.874166 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:42.870059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:56003)
0925 02:27:42.870181 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:42.870204 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:42.870265 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:42.870382 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:42.870383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:42.870524 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:42.870575 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:42.870576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.871168 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.871168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:42.871672 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:42.871673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.871972 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.871973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:42.872012 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:42.872508 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:42.872512 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:42.874094 (+ 1582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
W20260925 02:27:42.874279 21426 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:42.870128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:56003 (local address 127.11.220.254:40047)
0925 02:27:42.870273 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:42.870275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:42.870282 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:42.870302 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:42.870302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:42.870312 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:42.870368 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:42.870636 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.871106 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.871737 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:42.871913 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:42.871958 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:42.874200 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:56003: BlockingRecv error: recv got EOF from 127.11.220.193:56003 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
Sep 25 02:27:43 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:43 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:43.008560 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:43.004339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:52353)
0925 02:27:43.004517 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:43.004534 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:43.004589 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:43.004736 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:43.004736 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:43.004883 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:43.004931 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:43.004932 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.005524 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.005525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:43.006024 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:43.006025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.006306 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.006307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:43.006340 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:43.006781 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:43.006785 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:43.008486 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
W20260925 02:27:43.008647 21426 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:43.004397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:52353 (local address 127.11.220.254:40047)
0925 02:27:43.004489 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:43.004490 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:43.004587 (+ 97us) server_negotiation.cc:407] Connection header received
0925 02:27:43.004663 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:43.004664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:43.004673 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:43.004718 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:43.004984 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.005465 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.006083 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.006252 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.006293 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:43.008562 (+ 2269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:52353: BlockingRecv error: recv got EOF from 127.11.220.194:52353 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 25 02:27:43 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:43 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:43.135613 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:43.131252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:40339)
0925 02:27:43.131418 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:43.131429 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:43.131509 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:43.131656 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:43.131657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:43.131824 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:43.131870 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:43.131872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.132549 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.132550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:43.133033 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:43.133035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.133442 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.133443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:43.133507 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:43.133909 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:43.133913 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:43.135556 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
W20260925 02:27:43.135712 21426 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:43.131323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:40339 (local address 127.11.220.254:40047)
0925 02:27:43.131405 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:43.131407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:43.131493 (+ 86us) server_negotiation.cc:407] Connection header received
0925 02:27:43.131561 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:43.131562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:43.131581 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:43.131651 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:43.131947 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.132433 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.133159 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.133343 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.133428 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:43.135649 (+ 2221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:40339: BlockingRecv error: recv got EOF from 127.11.220.195:40339 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 02:27:43 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:43 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:43.878960 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:43.874620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:34775)
0925 02:27:43.874776 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:43.874790 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:43.874849 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:43.874975 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:43.874976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:43.875131 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:43.875180 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:43.875181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.875754 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.875754 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:43.876227 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:43.876228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.876583 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.876584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:43.876621 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:43.877102 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:43.877107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:43.878895 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
W20260925 02:27:43.879092 21430 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:43.874693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:34775 (local address 127.11.220.254:40047)
0925 02:27:43.874835 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:43.874837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:43.874844 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:43.874890 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:43.874891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:43.874900 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:43.874956 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:43.875235 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.875694 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.876286 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:43.876529 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:43.876575 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:43.878977 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:34775: BlockingRecv error: recv got EOF from 127.11.220.193:34775 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
Sep 25 02:27:44 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:44 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:44.013581 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:44.009032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:59673)
0925 02:27:44.009167 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:44.009177 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:44.009234 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:44.009347 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:44.009347 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:44.009531 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:44.009579 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:44.009580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.010404 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.010404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:44.010929 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:44.010930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.011222 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.011223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:44.011259 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:44.011662 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:44.011666 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:44.013522 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
W20260925 02:27:44.013651 21430 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:44.009047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:59673 (local address 127.11.220.254:40047)
0925 02:27:44.009089 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:44.009091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:44.009236 (+ 145us) server_negotiation.cc:407] Connection header received
0925 02:27:44.009272 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:44.009273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:44.009284 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:44.009328 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:44.009641 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.010344 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.010992 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.011159 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.011210 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:44.013607 (+ 2397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:59673: BlockingRecv error: recv got EOF from 127.11.220.194:59673 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 25 02:27:44 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:44 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:44.140569 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:44.136080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:35099)
0925 02:27:44.136217 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:44.136226 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:44.136279 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:44.136407 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:44.136408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:44.136613 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:44.136660 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:44.136662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.137278 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.137279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:44.137759 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:44.137761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.138093 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.138094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:44.138158 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:44.138541 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:44.138545 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:44.140419 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
W20260925 02:27:44.140594 21430 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:44.136136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:35099 (local address 127.11.220.254:40047)
0925 02:27:44.136200 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:44.136204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:44.136278 (+ 74us) server_negotiation.cc:407] Connection header received
0925 02:27:44.136318 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:44.136319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:44.136331 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:44.136389 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:44.136716 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.137202 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.137845 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.138025 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.138092 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:44.140548 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:35099: BlockingRecv error: recv got EOF from 127.11.220.195:35099 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 02:27:44 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:44 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:44.883150 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:44.879347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:38399)
0925 02:27:44.879464 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:44.879474 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:44.879525 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:44.879644 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:44.879645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:44.879789 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:44.879837 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:44.879838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.880399 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.880400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:44.880868 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:44.880869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.881189 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.881190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:44.881236 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:44.881589 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:44.881592 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:44.883097 (+ 1505us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
W20260925 02:27:44.883266 21434 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:44.879402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:38399 (local address 127.11.220.254:40047)
0925 02:27:44.879517 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:44.879519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:44.879525 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:27:44.879573 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:44.879574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:44.879583 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:44.879627 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:44.879890 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.880337 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.880929 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:44.881117 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:44.881165 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:44.883186 (+ 2021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:38399: BlockingRecv error: recv got EOF from 127.11.220.193:38399 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":59,"threads_started":1}
Sep 25 02:27:45 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:45 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:45.019032 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:45.014102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:51515)
0925 02:27:45.014238 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:45.014247 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:45.014325 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:45.014465 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:45.014466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:45.014615 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:45.014662 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:45.014664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.015688 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.015689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:45.016236 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:45.016238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.016560 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.016561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:45.016606 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:45.017001 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:45.017005 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:45.018969 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
W20260925 02:27:45.019152 21434 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:45.014181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:51515 (local address 127.11.220.254:40047)
0925 02:27:45.014241 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:45.014243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:45.014318 (+ 75us) server_negotiation.cc:407] Connection header received
0925 02:27:45.014377 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:45.014377 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:45.014387 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:45.014455 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:45.014734 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.015605 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.016305 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.016501 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.016546 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:45.019058 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:51515: BlockingRecv error: recv got EOF from 127.11.220.194:51515 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 25 02:27:45 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:45 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:45.150166 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:45.141088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:33095)
0925 02:27:45.141244 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:45.141259 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:45.141311 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:45.141433 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:45.141433 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:45.141583 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:45.141633 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:45.141635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.142209 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.142209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:45.142671 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:45.142673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.142974 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.142975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:45.143014 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:45.148348 (+ 5334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:45.148353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:45.150110 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
W20260925 02:27:45.150285 21434 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:45.141159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:33095 (local address 127.11.220.254:40047)
0925 02:27:45.141228 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:45.141230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:45.141320 (+ 90us) server_negotiation.cc:407] Connection header received
0925 02:27:45.141347 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:45.141347 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:45.141360 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:45.141412 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:45.141702 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.142143 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.142733 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.142915 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.142962 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:45.150193 (+ 7231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:33095: BlockingRecv error: recv got EOF from 127.11.220.195:33095 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 25 02:27:45 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:45 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:45.887904 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:45.883578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:53163)
0925 02:27:45.883710 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:45.883719 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:45.883762 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:45.883962 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:45.883963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:45.884136 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:45.884187 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:45.884188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.884853 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.884854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:45.885325 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:45.885326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.885618 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.885619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:45.885656 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:45.886263 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:45.886269 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:45.887847 (+ 1578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
W20260925 02:27:45.888007 21438 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:45.883739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:53163 (local address 127.11.220.254:40047)
0925 02:27:45.883852 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:45.883854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:45.883860 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:27:45.883880 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:45.883880 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:45.883889 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:45.883945 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:45.884246 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.884779 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.885385 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:45.885555 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:45.885599 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:45.887934 (+ 2335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:53163: BlockingRecv error: recv got EOF from 127.11.220.193:53163 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":64,"threads_started":1}
Sep 25 02:27:46 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:46 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:46.024004 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:46.019551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:34717)
0925 02:27:46.019717 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:46.019727 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:46.019781 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:46.019899 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:46.019900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:46.020063 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:46.020117 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:46.020119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.020737 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.020738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:46.021242 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:46.021243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.021538 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.021538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:46.021577 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:46.021987 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:46.021991 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:46.023949 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
W20260925 02:27:46.024096 21438 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:46.019598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:34717 (local address 127.11.220.254:40047)
0925 02:27:46.019670 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:46.019672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:46.019785 (+ 113us) server_negotiation.cc:407] Connection header received
0925 02:27:46.019825 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:46.019825 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:46.019836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:46.019884 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:46.020188 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.020675 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.021307 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.021478 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.021526 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:46.024030 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:34717: BlockingRecv error: recv got EOF from 127.11.220.194:34717 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 25 02:27:46 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:46 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:46.154855 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:46.150586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:59009)
0925 02:27:46.150712 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:46.150725 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:46.150806 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:46.150968 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:46.150968 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:46.151198 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:46.151259 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:46.151262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.152010 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.152010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:46.152523 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:46.152525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.152824 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.152825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:46.152875 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:46.153270 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:46.153274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:46.154800 (+ 1526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
W20260925 02:27:46.154922 21438 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:46.150744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:59009 (local address 127.11.220.254:40047)
0925 02:27:46.150843 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:46.150845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:46.150854 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:46.150877 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:46.150878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:46.150893 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:46.150958 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:46.151346 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.151940 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.152585 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.152761 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.152809 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:46.154880 (+ 2071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:59009: BlockingRecv error: recv got EOF from 127.11.220.195:59009 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 02:27:46 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:46 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:46.893106 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:46.888531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:52901)
0925 02:27:46.888667 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:46.888676 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:46.888724 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:46.888858 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:46.888858 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:46.889018 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:46.889067 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:46.889068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.889685 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.889686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:46.890195 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:46.890196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.890517 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.890518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:46.890557 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:46.891077 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:46.891081 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:46.893037 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
W20260925 02:27:46.893146 21442 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:46.888580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:52901 (local address 127.11.220.254:40047)
0925 02:27:46.888732 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:46.888734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:46.888743 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:46.888772 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:46.888772 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:46.888782 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:46.888842 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:46.889132 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.889616 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.890275 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:46.890457 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:46.890503 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:46.893094 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:52901: BlockingRecv error: recv got EOF from 127.11.220.193:52901 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
Sep 25 02:27:47 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:47 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:47.029597 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:47.024401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:53059)
0925 02:27:47.024571 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:47.024581 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:47.024658 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:47.024803 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:47.024804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:47.024976 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:47.025018 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:47.025020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.025664 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.025665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:47.026179 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:47.026181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.026523 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.026525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:47.026582 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:47.027059 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:47.027077 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:47.029479 (+ 2402us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
W20260925 02:27:47.029633 21442 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:47.024490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:53059 (local address 127.11.220.254:40047)
0925 02:27:47.024551 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:47.024554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:47.024643 (+ 89us) server_negotiation.cc:407] Connection header received
0925 02:27:47.024712 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:47.024713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:47.024723 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:47.024783 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:47.025088 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.025574 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.026257 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.026445 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.026523 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:47.029573 (+ 3050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:53059: BlockingRecv error: recv got EOF from 127.11.220.194:53059 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 25 02:27:47 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:47 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:47.159642 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:47.155559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:50421)
0925 02:27:47.155711 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:47.155722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:47.155786 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:47.155913 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:47.155914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:47.156067 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:47.156114 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:47.156116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.156776 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.156777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:47.157274 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:47.157276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.157541 (+ 265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.157541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:47.157577 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:47.157954 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:47.157958 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:47.159545 (+ 1587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
W20260925 02:27:47.159687 21442 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:47.155634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:50421 (local address 127.11.220.254:40047)
0925 02:27:47.155704 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:47.155706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:47.155780 (+ 74us) server_negotiation.cc:407] Connection header received
0925 02:27:47.155829 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:47.155830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:47.155841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:47.155892 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:47.156181 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.156716 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.157333 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.157485 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.157525 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:47.159615 (+ 2090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:50421: BlockingRecv error: recv got EOF from 127.11.220.195:50421 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 25 02:27:47 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:47 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:47.897720 21445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:47.893657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:47983)
0925 02:27:47.893797 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:47.893806 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:47.893852 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:47.893966 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:47.893967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:47.894115 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:47.894160 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:47.894162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.894727 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.894727 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:47.895196 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:47.895197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.895525 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.895526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:47.895567 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:47.895979 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:47.895983 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:47.897659 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
W20260925 02:27:47.897861 21446 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:47.893743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:47983 (local address 127.11.220.254:40047)
0925 02:27:47.893833 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:47.893835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:47.893843 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:47.893893 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:47.893893 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:47.893903 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:47.893953 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:47.894220 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.894669 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.895267 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:47.895446 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:47.895511 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:47.897771 (+ 2260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:47983: BlockingRecv error: recv got EOF from 127.11.220.193:47983 (error 108)
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":43,"threads_started":1}
Sep 25 02:27:48 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:48 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:48.034508 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:48.030244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:44851)
0925 02:27:48.030367 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:48.030376 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:48.030440 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:48.030584 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:48.030585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:48.030754 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:48.030796 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:48.030798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.031493 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.031494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:48.032001 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:48.032002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.032352 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.032353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:48.032411 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:48.032820 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:48.032824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:48.034414 (+ 1590us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
W20260925 02:27:48.034545 21446 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:48.030311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:44851 (local address 127.11.220.254:40047)
0925 02:27:48.030352 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:48.030354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:48.030444 (+ 90us) server_negotiation.cc:407] Connection header received
0925 02:27:48.030500 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:48.030501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:48.030512 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:48.030575 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:48.030864 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.031414 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.032095 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.032277 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.032349 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:48.034489 (+ 2140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:44851: BlockingRecv error: recv got EOF from 127.11.220.194:44851 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 02:27:48 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:48 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:48.164402 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:48.160144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:53483)
0925 02:27:48.160300 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:48.160312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:48.160393 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:48.160540 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:48.160541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:48.160740 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:48.160785 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:48.160787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.161428 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.161429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:48.161938 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:48.161940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.162253 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.162254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:48.162315 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:48.162717 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:48.162721 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:48.164314 (+ 1593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":79,"threads_started":1}
W20260925 02:27:48.164466 21446 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:48.160236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:53483 (local address 127.11.220.254:40047)
0925 02:27:48.160302 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:48.160304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:48.160377 (+ 73us) server_negotiation.cc:407] Connection header received
0925 02:27:48.160461 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:48.160462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:48.160474 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:48.160534 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:48.160862 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.161335 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.162017 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.162187 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.162254 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:48.164387 (+ 2133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:53483: BlockingRecv error: recv got EOF from 127.11.220.195:53483 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 25 02:27:48 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:48 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:48.902282 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:48.898147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:47893)
0925 02:27:48.898283 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:48.898292 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:48.898338 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:48.898470 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:48.898471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:48.898622 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:48.898668 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:48.898669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.899236 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.899236 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:48.899715 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:48.899716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.900001 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.900002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:48.900049 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:48.900492 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:48.900496 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:48.902184 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
W20260925 02:27:48.902326 21450 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:48.898220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:47893 (local address 127.11.220.254:40047)
0925 02:27:48.898313 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:48.898320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:48.898327 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:48.898391 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:48.898392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:48.898402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:48.898454 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:48.898725 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.899179 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.899777 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:48.899946 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:48.899990 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:48.902258 (+ 2268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:47893: BlockingRecv error: recv got EOF from 127.11.220.193:47893 (error 108)
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":42,"threads_started":1}
Sep 25 02:27:49 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:49 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:49.038883 21451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:49.034917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:60957)
0925 02:27:49.035079 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:49.035088 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:49.035137 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:49.035241 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:49.035241 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:49.035385 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:49.035430 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:49.035432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.036002 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.036002 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:49.036488 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:49.036489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.036762 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.036763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:49.036810 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:49.037222 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:49.037228 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:49.038830 (+ 1602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":51,"threads_started":1}
W20260925 02:27:49.038985 21450 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:49.035013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:60957 (local address 127.11.220.254:40047)
0925 02:27:49.035079 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:49.035081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:49.035123 (+ 42us) server_negotiation.cc:407] Connection header received
0925 02:27:49.035181 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:49.035182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:49.035190 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:49.035237 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:49.035478 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.035942 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.036539 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.036705 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.036753 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:49.038900 (+ 2147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:60957: BlockingRecv error: recv got EOF from 127.11.220.194:60957 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 02:27:49 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:49 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:49.169283 21452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:49.165101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:44257)
0925 02:27:49.165235 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:49.165244 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:49.165294 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:49.165408 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:49.165409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:49.165556 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:49.165604 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:49.165606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.166412 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.166414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:49.166946 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:49.166947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.167223 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.167224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:49.167260 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:49.167639 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:49.167642 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:49.169219 (+ 1577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
W20260925 02:27:49.169456 21450 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:49.165170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:44257 (local address 127.11.220.254:40047)
0925 02:27:49.165238 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:49.165240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:49.165298 (+ 58us) server_negotiation.cc:407] Connection header received
0925 02:27:49.165335 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:49.165335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:49.165346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:49.165393 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:49.165682 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.166351 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.167008 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.167167 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.167209 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:49.169329 (+ 2120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:44257: BlockingRecv error: recv got EOF from 127.11.220.195:44257 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 02:27:49 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:49 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:49.906893 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:49.902724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:43561)
0925 02:27:49.902861 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:49.902871 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:49.902934 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:49.903082 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:49.903083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:49.903239 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:49.903287 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:49.903289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.903869 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.903871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:49.904422 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:49.904425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.904785 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.904786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:49.904828 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:49.905203 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:49.905208 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:49.906837 (+ 1629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
W20260925 02:27:49.907009 21454 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:49.902768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:43561 (local address 127.11.220.254:40047)
0925 02:27:49.902888 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:49.902894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:49.902930 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:27:49.902974 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:49.902974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:49.902984 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:49.903064 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:49.903346 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.903812 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.904536 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:49.904725 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:49.904776 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:49.906927 (+ 2151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:43561: BlockingRecv error: recv got EOF from 127.11.220.193:43561 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
Sep 25 02:27:50 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:50 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:50.044118 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:50.039265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:53839)
0925 02:27:50.039437 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:50.039447 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:50.039515 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:50.039647 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:50.039648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:50.039811 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:50.039858 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:50.039860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.040674 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.040675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:50.041152 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:50.041154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.041643 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.041644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:50.041681 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:50.042119 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:50.042126 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:50.043998 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":53,"threads_started":1}
W20260925 02:27:50.044201 21454 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:50.039308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:53839 (local address 127.11.220.254:40047)
0925 02:27:50.039359 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:50.039362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:50.039502 (+ 140us) server_negotiation.cc:407] Connection header received
0925 02:27:50.039571 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:50.039572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:50.039585 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:50.039661 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:50.039914 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.040587 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.041241 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.041540 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.041650 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:50.044128 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:53839: BlockingRecv error: recv got EOF from 127.11.220.194:53839 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 25 02:27:50 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:50 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.195@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:50.179917 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:50.169797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.195:33727)
0925 02:27:50.169947 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:50.169956 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:50.170003 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:50.170136 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:50.170137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:50.170294 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:50.170340 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:50.170342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.171000 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.171000 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:50.171500 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:50.171502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.171780 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.171780 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:50.171816 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:50.178087 (+ 6271us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:50.178092 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:50.179863 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":76,"threads_started":1}
W20260925 02:27:50.180030 21454 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:50.169861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:33727 (local address 127.11.220.254:40047)
0925 02:27:50.169916 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:50.169917 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:50.170002 (+ 85us) server_negotiation.cc:407] Connection header received
0925 02:27:50.170057 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:50.170057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:50.170070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:50.170131 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:50.170392 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.170937 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.171565 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.171725 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.171769 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:50.179947 (+ 8178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.195:33727: BlockingRecv error: recv got EOF from 127.11.220.195:33727 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 25 02:27:50 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:50 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.193@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:50.911616 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:50.907380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.193:46885)
0925 02:27:50.907502 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:50.907515 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:50.907587 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:50.907737 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:50.907738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:50.907883 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:50.907952 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:50.907953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.908571 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.908572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:50.909091 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:50.909093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.909402 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.909403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:50.909469 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:50.909851 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:50.909855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:50.911555 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
W20260925 02:27:50.911734 21458 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:50.907455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:46885 (local address 127.11.220.254:40047)
0925 02:27:50.907586 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:50.907588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:50.907597 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:50.907631 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:50.907632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:50.907642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:50.907707 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:50.908014 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.908507 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.909154 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:50.909321 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:50.909393 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:50.911634 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.193:46885: BlockingRecv error: recv got EOF from 127.11.220.193:46885 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
Sep 25 02:27:51 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 02:27:51 dist-test-slave-dn4n krb5kdc[20377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.220.194@KRBTEST.COM for oryx/127.11.220.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 02:27:51.049521 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.044812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40047 (local address 127.11.220.194:50615)
0925 02:27:51.045054 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:51.045065 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:51.045126 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:51.045300 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:51.045301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:51.045465 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:51.045510 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:51.045512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.046187 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.046188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:51.046736 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:51.046738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.047124 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.047126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:51.047191 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:51.047642 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:51.047646 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:51.049418 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.220.254:40047: Server oryx/127.11.220.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":175,"threads_started":1}
W20260925 02:27:51.049553 21458 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:51.045002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:50615 (local address 127.11.220.254:40047)
0925 02:27:51.045048 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:51.045050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:51.045129 (+ 79us) server_negotiation.cc:407] Connection header received
0925 02:27:51.045199 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:51.045200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:51.045212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:51.045279 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:51.045587 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.046091 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.046814 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.047039 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.047124 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:51.049492 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.220.194:50615: BlockingRecv error: recv got EOF from 127.11.220.194:50615 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 02:27:51.097795 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 20970
I20260925 02:27:51.102487 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 21126
I20260925 02:27:51.107005 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 21262
I20260925 02:27:51.111269 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 20900
2026-09-25T02:27:51Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17219 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T02:27:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:51Z Disabled control of system clock
I20260925 02:27:51.137601 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:36741
--webserver_interface=127.11.220.254
--webserver_port=0
--builtin_ntp_servers=127.11.220.212:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:36741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 02:27:51.209311 21463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:51.209476 21463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:51.209493 21463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:51.210433 21463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:51.210460 21463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:51.210470 21463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:51.210481 21463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:51.210491 21463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:51.211548 21463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:36741
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:36741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:51.211695 21463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:51.211879 21463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:51.215957 21469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.215977 21471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.215977 21468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:51.215997 21463 server_base.cc:1034] running on GCE node
I20260925 02:27:51.216387 21463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:51.216715 21463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:51.217859 21463 hybrid_clock.cc:648] HybridClock initialized: now 1790303271217840 us; error 36 us; skew 500 ppm
I20260925 02:27:51.219060 21463 webserver.cc:460] Webserver started at http://127.11.220.254:40493/ using document root <none> and password file <none>
I20260925 02:27:51.219286 21463 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:51.219332 21463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:51.219482 21463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:51.220580 21463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "23810382bfc94a8cba43f62bfb75cbe9"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "2a596e8c7be78fff58fa7045b2a90594"
server_key_iv: "df4435de2b78fd1b27a8b8a7b53096a9"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.220933 21463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "23810382bfc94a8cba43f62bfb75cbe9"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "2a596e8c7be78fff58fa7045b2a90594"
server_key_iv: "df4435de2b78fd1b27a8b8a7b53096a9"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.222095 21463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:51.222769 21477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.222924 21463 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.223021 21463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "23810382bfc94a8cba43f62bfb75cbe9"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "2a596e8c7be78fff58fa7045b2a90594"
server_key_iv: "df4435de2b78fd1b27a8b8a7b53096a9"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.223086 21463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:51.233332 21463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:51.233551 21463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:51.233682 21463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:51.237110 21463 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:36741
I20260925 02:27:51.237133 21530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:36741 every 8 connection(s)
I20260925 02:27:51.237406 21463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:51.237999 21531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:51.240123 21531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9: Bootstrap starting.
I20260925 02:27:51.240680 21531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:51.240912 21531 log.cc:824] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:51.241173 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 21463
I20260925 02:27:51.241294 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:51.241462 12147 external_mini_cluster.cc:1428] Setting key 007344a651cda5d572d05a6f98832fbe
I20260925 02:27:51.241600 21531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9: No bootstrap required, opened a new log
I20260925 02:27:51.243124 21531 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23810382bfc94a8cba43f62bfb75cbe9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36741 } }
I20260925 02:27:51.243237 21531 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:51.243310 21531 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23810382bfc94a8cba43f62bfb75cbe9, State: Initialized, Role: FOLLOWER
I20260925 02:27:51.243462 21531 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23810382bfc94a8cba43f62bfb75cbe9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36741 } }
I20260925 02:27:51.243553 21531 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:51.243577 21531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:51.243639 21531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:51.244112 21531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23810382bfc94a8cba43f62bfb75cbe9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36741 } }
I20260925 02:27:51.244190 21531 leader_election.cc:302] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23810382bfc94a8cba43f62bfb75cbe9; no voters:
I20260925 02:27:51.244366 21531 leader_election.cc:288] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:51.244396 21537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:51.244515 21537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [term 1 LEADER]: Becoming Leader. State: Replica: 23810382bfc94a8cba43f62bfb75cbe9, State: Running, Role: LEADER
I20260925 02:27:51.244618 21531 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:51.244647 21537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23810382bfc94a8cba43f62bfb75cbe9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36741 } }
I20260925 02:27:51.245013 21537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23810382bfc94a8cba43f62bfb75cbe9. Latest consensus state: current_term: 1 leader_uuid: "23810382bfc94a8cba43f62bfb75cbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23810382bfc94a8cba43f62bfb75cbe9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36741 } } }
I20260925 02:27:51.245267 21537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:51.245193 21539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23810382bfc94a8cba43f62bfb75cbe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23810382bfc94a8cba43f62bfb75cbe9" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36741 } } }
I20260925 02:27:51.245419 21539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:51.245790 21546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:51.246286 21546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:51.248013 21546 catalog_manager.cc:1347] Generated new cluster ID: f49c2fdcca1d460ab225f9a0a67c88f3
I20260925 02:27:51.248082 21546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:51.248248 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.241871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.11.220.254:36741)
0925 02:27:51.242029 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:51.242041 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:51.242061 (+ 20us) server_negotiation.cc:407] Connection header received
0925 02:27:51.243114 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:51.243119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:51.243176 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:51.243302 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:51.243525 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.245034 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.246187 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.246736 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.247132 (+ 396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:51.247514 (+ 382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:51.247518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:51.247518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:51.247697 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:51.247735 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:51.247749 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:51.247819 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:51.247949 (+ 130us) server_negotiation.cc:299] Negotiation successful
0925 02:27:51.248026 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260925 02:27:51.274386 21546 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:51.274982 21546 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:51.280378 21546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23810382bfc94a8cba43f62bfb75cbe9: Generated new TSK 0
I20260925 02:27:51.280557 21546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:27:51.282095 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36741
--builtin_ntp_servers=127.11.220.212:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 02:27:51.362059 21557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:51.362221 21557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:51.362238 21557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:51.363198 21557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:51.363230 21557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:51.363250 21557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:51.364435 21557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:51.364692 21557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:51.364951 21557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:51.369388 21565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.369505 21563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.369462 21562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:51.369614 21557 server_base.cc:1034] running on GCE node
I20260925 02:27:51.369784 21557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:51.369940 21557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:51.371104 21557 hybrid_clock.cc:648] HybridClock initialized: now 1790303271371086 us; error 43 us; skew 500 ppm
I20260925 02:27:51.372226 21557 webserver.cc:460] Webserver started at http://127.11.220.193:40783/ using document root <none> and password file <none>
I20260925 02:27:51.372521 21557 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:51.372598 21557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:51.372733 21557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:51.373697 21557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "933ec07624dc4e3eaead99af36ea0aa7"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "2d84c7f4d9b705b05d28bb8286a97579"
server_key_iv: "3750d10f0ee5dd0b40af1b105fefe2b7"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.374023 21557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "933ec07624dc4e3eaead99af36ea0aa7"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "2d84c7f4d9b705b05d28bb8286a97579"
server_key_iv: "3750d10f0ee5dd0b40af1b105fefe2b7"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.375226 21557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:27:51.375921 21571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.376087 21557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:51.376188 21557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "933ec07624dc4e3eaead99af36ea0aa7"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "2d84c7f4d9b705b05d28bb8286a97579"
server_key_iv: "3750d10f0ee5dd0b40af1b105fefe2b7"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.376273 21557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:51.395301 21557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:51.395531 21557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:51.395661 21557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:51.395869 21557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:51.396157 21557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:51.396207 21557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.396245 21557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:51.396270 21557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.401446 21557 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:39911
I20260925 02:27:51.401489 21685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:39911 every 8 connection(s)
I20260925 02:27:51.401746 21557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:51.406538 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.402379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:60645 (local address 127.11.220.254:36741)
0925 02:27:51.402436 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:51.402449 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:51.402666 (+ 217us) server_negotiation.cc:407] Connection header received
0925 02:27:51.403149 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:51.403150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:51.403165 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:51.403225 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:51.403838 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.404299 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.405280 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.405477 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.405549 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:51.406259 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:51.406261 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:51.406261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:51.406266 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:51.406280 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:51.406284 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:51.406338 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:51.406432 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 02:27:51.406462 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 02:27:51.406747 21689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.402375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:36741 (local address 127.11.220.193:60645)
0925 02:27:51.402564 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:51.402591 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:51.403044 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:51.403288 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:51.403291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:51.403301 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:51.403744 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:51.403748 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.404393 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.404394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:51.405180 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:51.405182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.405568 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.405569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:51.405695 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:51.406130 (+ 435us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:51.406130 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:51.406155 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:51.406174 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:51.406345 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:51.406350 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:51.406353 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:51.406433 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 02:27:51.406528 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260925 02:27:51.406863 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 21557
I20260925 02:27:51.406966 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:51.407145 12147 external_mini_cluster.cc:1428] Setting key 07aeeddef39d2f9a770291a8ac835f53
I20260925 02:27:51.407212 21686 heartbeater.cc:347] Connected to a master server at 127.11.220.254:36741
I20260925 02:27:51.407302 21686 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:51.407477 21686 heartbeater.cc:511] Master 127.11.220.254:36741 requested a full tablet report, sending...
I20260925 02:27:51.408001 21494 ts_manager.cc:194] Registered new tserver with Master: 933ec07624dc4e3eaead99af36ea0aa7 (127.11.220.193:39911)
I20260925 02:27:51.408097 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36741
--builtin_ntp_servers=127.11.220.212:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 02:27:51.408878 21494 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.193:60645
W20260925 02:27:51.479594 21690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:51.479759 21690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:51.479779 21690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:51.480808 21690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:51.480839 21690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:51.480860 21690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:51.482045 21690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:51.482198 21690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:51.482414 21690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:51.486649 21698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.486656 21695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.486656 21696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:51.486846 21690 server_base.cc:1034] running on GCE node
I20260925 02:27:51.487082 21690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:51.487301 21690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:51.488483 21690 hybrid_clock.cc:648] HybridClock initialized: now 1790303271488466 us; error 25 us; skew 500 ppm
I20260925 02:27:51.489595 21690 webserver.cc:460] Webserver started at http://127.11.220.194:33083/ using document root <none> and password file <none>
I20260925 02:27:51.489817 21690 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:51.489878 21690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:51.490000 21690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:51.490871 21690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "fd355922e04e4a458553ad2e55b85c14"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "09d5c0f5f7b7deed2b35d56356ab2998"
server_key_iv: "7b77ac3e69a492c1ad1e4527d92fc1a6"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.491180 21690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "fd355922e04e4a458553ad2e55b85c14"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "09d5c0f5f7b7deed2b35d56356ab2998"
server_key_iv: "7b77ac3e69a492c1ad1e4527d92fc1a6"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.492290 21690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:51.492987 21704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.493144 21690 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.493228 21690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "fd355922e04e4a458553ad2e55b85c14"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "09d5c0f5f7b7deed2b35d56356ab2998"
server_key_iv: "7b77ac3e69a492c1ad1e4527d92fc1a6"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.493309 21690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:51.501293 21690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:51.501514 21690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:51.501641 21690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:51.501816 21690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:51.502115 21690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:51.502166 21690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.502202 21690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:51.502224 21690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.507539 21690 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:38123
I20260925 02:27:51.507599 21819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:38123 every 8 connection(s)
I20260925 02:27:51.507869 21690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:51.512274 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 21690
I20260925 02:27:51.512368 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:51.512493 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.508557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:50131 (local address 127.11.220.254:36741)
0925 02:27:51.508617 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:51.508619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:51.508831 (+ 212us) server_negotiation.cc:407] Connection header received
0925 02:27:51.509270 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:51.509272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:51.509283 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:51.509339 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:51.509944 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.510424 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.511317 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.511506 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.511580 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:51.512219 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:51.512220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:51.512221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:51.512224 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:51.512238 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:51.512242 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:51.512281 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:51.512377 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 02:27:51.512404 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 02:27:51.512555 12147 external_mini_cluster.cc:1428] Setting key 23ffeadfdd9df4c7011fff497c8103b2
I20260925 02:27:51.512836 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.508568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:36741 (local address 127.11.220.194:50131)
0925 02:27:51.508731 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:51.508758 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:51.509173 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:51.509395 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:51.509399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:51.509408 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:51.509856 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:51.509860 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.510511 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.510513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:51.511226 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:51.511227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.511591 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.511592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:51.511713 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:51.512069 (+ 356us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:51.512069 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:51.512090 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:51.512140 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:51.512283 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:51.512287 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:51.512289 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:51.512570 (+ 281us) client_negotiation.cc:240] Negotiation successful
0925 02:27:51.512664 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260925 02:27:51.513177 21820 heartbeater.cc:347] Connected to a master server at 127.11.220.254:36741
I20260925 02:27:51.513252 21820 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:51.513405 21820 heartbeater.cc:511] Master 127.11.220.254:36741 requested a full tablet report, sending...
I20260925 02:27:51.513440 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36741
--builtin_ntp_servers=127.11.220.212:43651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 02:27:51.513697 21494 ts_manager.cc:194] Registered new tserver with Master: fd355922e04e4a458553ad2e55b85c14 (127.11.220.194:38123)
I20260925 02:27:51.514014 21494 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.194:50131
W20260925 02:27:51.585973 21823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:51.586145 21823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:51.586165 21823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:51.587294 21823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:51.587327 21823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:51.587349 21823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:51.588665 21823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:51.588874 21823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:51.589179 21823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:51.593639 21828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:51.593726 21823 server_base.cc:1034] running on GCE node
W20260925 02:27:51.593648 21831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.593683 21829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:51.594046 21823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:51.594292 21823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:51.595439 21823 hybrid_clock.cc:648] HybridClock initialized: now 1790303271595434 us; error 30 us; skew 500 ppm
I20260925 02:27:51.596606 21823 webserver.cc:460] Webserver started at http://127.11.220.195:42219/ using document root <none> and password file <none>
I20260925 02:27:51.596828 21823 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:51.596891 21823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:51.597033 21823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:51.597919 21823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "31e0c35bcd9642c8b6d0c27c0f5c4176"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "bc5af94046a2d1764856dfc90b9e0212"
server_key_iv: "67a9f01d5c2ed175647bf384c398aa33"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.598240 21823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "31e0c35bcd9642c8b6d0c27c0f5c4176"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "bc5af94046a2d1764856dfc90b9e0212"
server_key_iv: "67a9f01d5c2ed175647bf384c398aa33"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.599337 21823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:27:51.599983 21837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.600169 21823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:51.600260 21823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "31e0c35bcd9642c8b6d0c27c0f5c4176"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "bc5af94046a2d1764856dfc90b9e0212"
server_key_iv: "67a9f01d5c2ed175647bf384c398aa33"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.600340 21823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:51.608721 21823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:51.608943 21823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:51.609076 21823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:51.609256 21823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:51.609512 21823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:51.609558 21823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.609591 21823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:51.609615 21823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.614480 21823 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:33823
I20260925 02:27:51.614544 21956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:33823 every 8 connection(s)
I20260925 02:27:51.614795 21823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:51.617583 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 21823
I20260925 02:27:51.617667 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:51.617851 12147 external_mini_cluster.cc:1428] Setting key 9670d36a6c88fb5c627cf5e321b42838
I20260925 02:27:51.619827 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.615357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:35861 (local address 127.11.220.254:36741)
0925 02:27:51.615392 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:51.615394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:51.615582 (+ 188us) server_negotiation.cc:407] Connection header received
0925 02:27:51.616065 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:51.616067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:51.616081 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:51.616149 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:51.616842 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.617391 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.618689 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.618934 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.618983 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:51.619592 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:51.619594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:51.619594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:51.619603 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:51.619616 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:51.619619 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:51.619657 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:51.619733 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 02:27:51.619756 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 02:27:51.620035 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.615331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:36741 (local address 127.11.220.195:35861)
0925 02:27:51.615472 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:51.615496 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:51.615963 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:51.616206 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:51.616210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:51.616218 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:51.616744 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:51.616749 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.617489 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.617490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:51.618613 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:51.618616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:51.619002 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:51.619002 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:51.619097 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:51.619477 (+ 380us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:51.619478 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:51.619500 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:51.619532 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:51.619674 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:51.619678 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:51.619680 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:51.619725 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 02:27:51.619810 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":58,"threads_started":1}
I20260925 02:27:51.620551 21957 heartbeater.cc:347] Connected to a master server at 127.11.220.254:36741
I20260925 02:27:51.620620 21957 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:51.620777 21957 heartbeater.cc:511] Master 127.11.220.254:36741 requested a full tablet report, sending...
I20260925 02:27:51.621066 21494 ts_manager.cc:194] Registered new tserver with Master: 31e0c35bcd9642c8b6d0c27c0f5c4176 (127.11.220.195:33823)
I20260925 02:27:51.621380 21494 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.195:35861
I20260925 02:27:51.629447 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 02:27:51.631215 21536 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:51.630703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53868 (local address 127.11.220.254:36741)
0925 02:27:51.630748 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:51.630750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:51.630867 (+ 117us) server_negotiation.cc:407] Connection header received
0925 02:27:51.630928 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:51.630929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:51.630941 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:51.631006 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:51.631150 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53868: BlockingRecv error: recv got EOF from 127.0.0.1:53868 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 02:27:51.631313 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 21557
I20260925 02:27:51.635938 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 21690
I20260925 02:27:51.639873 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 21823
I20260925 02:27:51.643934 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 21463
2026-09-25T02:27:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (532 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T02:27:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:51Z Disabled control of system clock
I20260925 02:27:51.669405 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:40595
--webserver_interface=127.11.220.254
--webserver_port=0
--builtin_ntp_servers=127.11.220.212:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:40595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 02:27:51.741636 21968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:51.741801 21968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:51.741818 21968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:51.742904 21968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:51.742940 21968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:51.742952 21968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:51.742964 21968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:51.743007 21968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:51.744305 21968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:40595
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:40595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:51.744481 21968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:51.744735 21968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:51.748845 21974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.748862 21973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:51.748883 21968 server_base.cc:1034] running on GCE node
W20260925 02:27:51.748857 21976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:51.749194 21968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:51.749468 21968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:51.750598 21968 hybrid_clock.cc:648] HybridClock initialized: now 1790303271750591 us; error 27 us; skew 500 ppm
I20260925 02:27:51.751745 21968 webserver.cc:460] Webserver started at http://127.11.220.254:42155/ using document root <none> and password file <none>
I20260925 02:27:51.751962 21968 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:51.752004 21968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:51.752146 21968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:51.753209 21968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "9573389d56c74f63aba6bcda75c94903"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "5fa48310c58408d1656f775b0d561534"
server_key_iv: "4d12a6300631ec1c185a230e2b02af1d"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.753530 21968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "9573389d56c74f63aba6bcda75c94903"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "5fa48310c58408d1656f775b0d561534"
server_key_iv: "4d12a6300631ec1c185a230e2b02af1d"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.754665 21968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:51.755308 21982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.755471 21968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:51.755564 21968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "9573389d56c74f63aba6bcda75c94903"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "5fa48310c58408d1656f775b0d561534"
server_key_iv: "4d12a6300631ec1c185a230e2b02af1d"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.755684 21968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:51.758905 21968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:51.759137 21968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:51.759281 21968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:51.762852 21968 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:40595
I20260925 02:27:51.762879 22036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:40595 every 8 connection(s)
I20260925 02:27:51.763137 21968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:51.763734 22037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:51.766026 22037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903: Bootstrap starting.
I20260925 02:27:51.766498 22037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:51.766722 22037 log.cc:824] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:51.767325 22037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903: No bootstrap required, opened a new log
I20260925 02:27:51.768724 22037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9573389d56c74f63aba6bcda75c94903" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40595 } }
I20260925 02:27:51.768836 22037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:51.768854 22037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9573389d56c74f63aba6bcda75c94903, State: Initialized, Role: FOLLOWER
I20260925 02:27:51.768962 22037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9573389d56c74f63aba6bcda75c94903" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40595 } }
I20260925 02:27:51.769028 22037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:51.769048 22037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:51.769073 22037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:51.769490 22037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9573389d56c74f63aba6bcda75c94903" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40595 } }
I20260925 02:27:51.769562 22037 leader_election.cc:302] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9573389d56c74f63aba6bcda75c94903; no voters:
I20260925 02:27:51.769661 22037 leader_election.cc:288] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:51.769721 22040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:51.769893 22040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [term 1 LEADER]: Becoming Leader. State: Replica: 9573389d56c74f63aba6bcda75c94903, State: Running, Role: LEADER
I20260925 02:27:51.769903 22037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:51.769987 22040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9573389d56c74f63aba6bcda75c94903" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40595 } }
I20260925 02:27:51.770327 22041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9573389d56c74f63aba6bcda75c94903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9573389d56c74f63aba6bcda75c94903" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40595 } } }
I20260925 02:27:51.770479 22041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:51.770543 22042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9573389d56c74f63aba6bcda75c94903. Latest consensus state: current_term: 1 leader_uuid: "9573389d56c74f63aba6bcda75c94903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9573389d56c74f63aba6bcda75c94903" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40595 } } }
I20260925 02:27:51.770608 22042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903 [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:51.770931 22049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:51.771380 22049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:27:51.772814 22049 catalog_manager.cc:1347] Generated new cluster ID: 3854b4fdcb9149a4bea747738c4ceafa
I20260925 02:27:51.772861 22049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:51.772854 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 21968
I20260925 02:27:51.772929 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:51.773152 12147 external_mini_cluster.cc:1428] Setting key 758ea93aefae22fb4f455d71277c3f1e
I20260925 02:27:51.775157 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.773639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56572 (local address 127.11.220.254:40595)
0925 02:27:51.773788 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:51.773795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:51.773820 (+ 25us) server_negotiation.cc:407] Connection header received
0925 02:27:51.773918 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:51.773924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:51.773960 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:51.774071 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:51.774674 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:51.774677 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:51.774678 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:51.774681 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:51.774719 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:51.774730 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:51.774783 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:51.774867 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 02:27:51.774989 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260925 02:27:51.786639 22049 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:51.786775 22049 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:51.790248 22049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9573389d56c74f63aba6bcda75c94903: Generated new TSK 0
I20260925 02:27:51.790396 22049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:27:51.795491 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40595
--builtin_ntp_servers=127.11.220.212:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 02:27:51.876261 22061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:51.876487 22061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:51.876530 22061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:51.877607 22061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:51.877663 22061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:51.877702 22061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:51.878913 22061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:51.879140 22061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:51.879412 22061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:27:51.884642 22061 server_base.cc:1034] running on GCE node
W20260925 02:27:51.884662 22066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.884681 22069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.884665 22067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:51.885030 22061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:51.885277 22061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:51.886442 22061 hybrid_clock.cc:648] HybridClock initialized: now 1790303271886431 us; error 26 us; skew 500 ppm
I20260925 02:27:51.887506 22061 webserver.cc:460] Webserver started at http://127.11.220.193:45023/ using document root <none> and password file <none>
I20260925 02:27:51.887718 22061 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:51.887778 22061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:51.887914 22061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:51.888851 22061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "71c99ff501374fa19a01266c01b31591"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "b14d041a8fa24ba0c692af4ce96123b4"
server_key_iv: "25246d4c8ea9f2ceb08fa977830480d5"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.889166 22061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "71c99ff501374fa19a01266c01b31591"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "b14d041a8fa24ba0c692af4ce96123b4"
server_key_iv: "25246d4c8ea9f2ceb08fa977830480d5"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.890522 22061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:51.891310 22075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.891482 22061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:27:51.891569 22061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "71c99ff501374fa19a01266c01b31591"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "b14d041a8fa24ba0c692af4ce96123b4"
server_key_iv: "25246d4c8ea9f2ceb08fa977830480d5"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.891644 22061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:51.894537 22061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:51.894767 22061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:51.894897 22061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:51.895080 22061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:51.895351 22061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:51.895401 22061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.895433 22061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:51.895459 22061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.900653 22061 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:43557
I20260925 02:27:51.900712 22191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:43557 every 8 connection(s)
I20260925 02:27:51.900961 22061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:27:51.902841 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 22061
I20260925 02:27:51.902946 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:51.903122 12147 external_mini_cluster.cc:1428] Setting key 9b672e30a588618aecb88566c34b099e
I20260925 02:27:51.903609 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.901492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:33797 (local address 127.11.220.254:40595)
0925 02:27:51.901572 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:51.901574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:51.901824 (+ 250us) server_negotiation.cc:407] Connection header received
0925 02:27:51.902215 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:51.902216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:51.902220 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:51.902268 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:51.903389 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:51.903390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:51.903390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:51.903394 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:51.903409 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:51.903413 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:51.903461 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:51.903534 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 02:27:51.903546 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 02:27:51.903797 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:51.901491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40595 (local address 127.11.220.193:33797)
0925 02:27:51.901698 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:51.901736 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:51.902133 (+ 397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:51.902328 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:51.902331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:51.902338 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:51.903238 (+ 900us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:51.903238 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:51.903269 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:51.903301 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:51.903464 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:51.903470 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:51.903473 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:51.903530 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 02:27:51.903576 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260925 02:27:51.904088 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40595
--builtin_ntp_servers=127.11.220.212:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 02:27:51.904263 22192 heartbeater.cc:347] Connected to a master server at 127.11.220.254:40595
I20260925 02:27:51.904345 22192 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:51.904495 22192 heartbeater.cc:511] Master 127.11.220.254:40595 requested a full tablet report, sending...
I20260925 02:27:51.904937 22000 ts_manager.cc:194] Registered new tserver with Master: 71c99ff501374fa19a01266c01b31591 (127.11.220.193:43557)
W20260925 02:27:51.976871 22195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:51.977062 22195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:51.977092 22195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:51.978093 22195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:51.978145 22195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:51.978180 22195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:51.979395 22195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:51.979648 22195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:51.979898 22195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:51.984134 22200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:51.984143 22203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:51.984253 22195 server_base.cc:1034] running on GCE node
W20260925 02:27:51.984347 22201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:51.984570 22195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:51.984790 22195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:51.985939 22195 hybrid_clock.cc:648] HybridClock initialized: now 1790303271985931 us; error 23 us; skew 500 ppm
I20260925 02:27:51.986927 22195 webserver.cc:460] Webserver started at http://127.11.220.194:46759/ using document root <none> and password file <none>
I20260925 02:27:51.987151 22195 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:51.987200 22195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:51.987354 22195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:51.988255 22195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "8ea497cf6f9e48c099e94ce715f45e36"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "6b72c2b7dc0259f517db6d0560622619"
server_key_iv: "b72fa9f9e3fe5446bb717e0e3fda969e"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.988601 22195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "8ea497cf6f9e48c099e94ce715f45e36"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "6b72c2b7dc0259f517db6d0560622619"
server_key_iv: "b72fa9f9e3fe5446bb717e0e3fda969e"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.989732 22195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:51.990443 22209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.990598 22195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:51.990685 22195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "8ea497cf6f9e48c099e94ce715f45e36"
format_stamp: "Formatted at 2026-09-25 02:27:51 on dist-test-slave-dn4n"
server_key: "6b72c2b7dc0259f517db6d0560622619"
server_key_iv: "b72fa9f9e3fe5446bb717e0e3fda969e"
server_key_version: "encryptionkey@0"
I20260925 02:27:51.990767 22195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:51.994345 22195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:51.994628 22195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:51.994793 22195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:51.995033 22195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:51.995411 22195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:51.995452 22195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:51.995496 22195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:51.995523 22195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:52.001550 22195 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:35579
I20260925 02:27:52.001619 22325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:35579 every 8 connection(s)
I20260925 02:27:52.001895 22195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:52.004271 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:52.002532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:60503 (local address 127.11.220.254:40595)
0925 02:27:52.002593 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:52.002594 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:52.002772 (+ 178us) server_negotiation.cc:407] Connection header received
0925 02:27:52.003146 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:52.003147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:52.003151 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:52.003200 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:52.004092 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:52.004093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:52.004094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:52.004103 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:52.004119 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:52.004122 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:52.004163 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:52.004212 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 02:27:52.004224 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 02:27:52.004424 22328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:52.002515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40595 (local address 127.11.220.194:60503)
0925 02:27:52.002665 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:52.002699 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:52.003068 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:52.003241 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:52.003243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:52.003250 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:52.003977 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:52.003977 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:52.003999 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:52.004022 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:52.004169 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:52.004171 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:52.004173 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:52.004217 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 02:27:52.004256 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
I20260925 02:27:52.004848 22326 heartbeater.cc:347] Connected to a master server at 127.11.220.254:40595
I20260925 02:27:52.004935 22326 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:52.005055 22326 heartbeater.cc:511] Master 127.11.220.254:40595 requested a full tablet report, sending...
I20260925 02:27:52.005362 22000 ts_manager.cc:194] Registered new tserver with Master: 8ea497cf6f9e48c099e94ce715f45e36 (127.11.220.194:35579)
I20260925 02:27:52.008684 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 22195
I20260925 02:27:52.008782 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:52.008947 12147 external_mini_cluster.cc:1428] Setting key 4158e89df62873df3df1472f4a480c33
I20260925 02:27:52.009734 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40595
--builtin_ntp_servers=127.11.220.212:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 02:27:52.082968 22329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:52.083169 22329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:52.083204 22329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:52.084288 22329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:52.084345 22329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:52.084388 22329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:52.085671 22329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:40595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:52.085906 22329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:52.086185 22329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:52.090727 22337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:52.090847 22329 server_base.cc:1034] running on GCE node
W20260925 02:27:52.090734 22335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:52.090921 22334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:52.091158 22329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:52.091351 22329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:52.092504 22329 hybrid_clock.cc:648] HybridClock initialized: now 1790303272092503 us; error 35 us; skew 500 ppm
I20260925 02:27:52.093541 22329 webserver.cc:460] Webserver started at http://127.11.220.195:43177/ using document root <none> and password file <none>
I20260925 02:27:52.093710 22329 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:52.093777 22329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:52.093925 22329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:52.094810 22329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "1455c1a3fa18404f9f65e1d41fbb8eef"
format_stamp: "Formatted at 2026-09-25 02:27:52 on dist-test-slave-dn4n"
server_key: "4a4261adb80194c24a02878d6c3b61a2"
server_key_iv: "ba57c4f33dd60f4844801a5127934df4"
server_key_version: "encryptionkey@0"
I20260925 02:27:52.095182 22329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "1455c1a3fa18404f9f65e1d41fbb8eef"
format_stamp: "Formatted at 2026-09-25 02:27:52 on dist-test-slave-dn4n"
server_key: "4a4261adb80194c24a02878d6c3b61a2"
server_key_iv: "ba57c4f33dd60f4844801a5127934df4"
server_key_version: "encryptionkey@0"
I20260925 02:27:52.096321 22329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 02:27:52.097018 22343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:52.097200 22329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:52.097277 22329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "1455c1a3fa18404f9f65e1d41fbb8eef"
format_stamp: "Formatted at 2026-09-25 02:27:52 on dist-test-slave-dn4n"
server_key: "4a4261adb80194c24a02878d6c3b61a2"
server_key_iv: "ba57c4f33dd60f4844801a5127934df4"
server_key_version: "encryptionkey@0"
I20260925 02:27:52.097343 22329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:52.100214 22329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:52.100483 22329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:52.100596 22329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:52.100782 22329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:52.101044 22329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:52.101071 22329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:52.101118 22329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:52.101145 22329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:52.106676 22329 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:34889
I20260925 02:27:52.106725 22459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:34889 every 8 connection(s)
I20260925 02:27:52.106993 22329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:27:52.109582 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:52.107627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:39333 (local address 127.11.220.254:40595)
0925 02:27:52.107669 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:52.107671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:52.107865 (+ 194us) server_negotiation.cc:407] Connection header received
0925 02:27:52.108323 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:52.108325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:52.108333 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:52.108378 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:52.109383 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:52.109385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:52.109385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:27:52.109390 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:52.109407 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:27:52.109412 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:52.109471 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:52.109518 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 02:27:52.109532 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:27:52.109750 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:52.107577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:40595 (local address 127.11.220.195:39333)
0925 02:27:52.107749 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:52.107796 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:52.108217 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:52.108434 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:52.108437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:52.108459 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:52.109208 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:27:52.109209 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:52.109246 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:27:52.109268 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:52.109471 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:52.109474 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:52.109475 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 02:27:52.109518 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 02:27:52.109558 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260925 02:27:52.110158 22460 heartbeater.cc:347] Connected to a master server at 127.11.220.254:40595
I20260925 02:27:52.110241 22460 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:52.110391 22460 heartbeater.cc:511] Master 127.11.220.254:40595 requested a full tablet report, sending...
I20260925 02:27:52.110729 22000 ts_manager.cc:194] Registered new tserver with Master: 1455c1a3fa18404f9f65e1d41fbb8eef (127.11.220.195:34889)
I20260925 02:27:52.113590 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 22329
I20260925 02:27:52.113682 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:52.113840 12147 external_mini_cluster.cc:1428] Setting key 60684b87922bbee86028ada746114b88
I20260925 02:27:52.114477 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 02:27:52.116063 22060 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:27:52.115665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56588 (local address 127.11.220.254:40595)
0925 02:27:52.115719 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:52.115721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:52.115822 (+ 101us) server_negotiation.cc:407] Connection header received
0925 02:27:52.115858 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:52.115858 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:52.115862 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:52.115906 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:52.116010 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56588: BlockingRecv error: recv got EOF from 127.0.0.1:56588 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:27:52.116137 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 22061
I20260925 02:27:52.122969 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 22195
I20260925 02:27:52.127063 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 22329
I20260925 02:27:52.131153 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 21968
2026-09-25T02:27:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (486 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:52 dist-test-slave-dn4n krb5kdc[22471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:52 dist-test-slave-dn4n krb5kdc[22471](info): set up 2 sockets
Sep 25 02:27:52 dist-test-slave-dn4n krb5kdc[22471](info): commencing operation
krb5kdc: starting...
W20260925 02:27:53.082753 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[22471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:27:53.182572 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:37301
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:37301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:53.255560 22485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:53.255800 22485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:53.255831 22485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:53.256930 22485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:27:53.256994 22485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:53.257015 22485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:53.257036 22485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:27:53.257066 22485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:27:53.258577 22485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:37301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:37301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:53.258854 22485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:53.259079 22485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:53.263936 22491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:53.263947 22490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:53.263998 22485 server_base.cc:1034] running on GCE node
W20260925 02:27:53.263952 22493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:53.264426 22485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:53.264791 22485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:53.265976 22485 hybrid_clock.cc:648] HybridClock initialized: now 1790303273265968 us; error 35 us; skew 500 ppm
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[22471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303273, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:53.268371 22485 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:27:53.268808 22485 webserver.cc:460] Webserver started at http://127.11.220.254:41855/ using document root <none> and password file <none>
I20260925 02:27:53.269030 22485 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:53.269095 22485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:53.269232 22485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:53.270269 22485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "7526b131e1574fd7a8b4c70f9c0bab3f"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "cb49cdb7c41499673d037f1efb998f82"
server_key_iv: "21df3de45b3ac84005f6ae3d1108748b"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.270645 22485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "7526b131e1574fd7a8b4c70f9c0bab3f"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "cb49cdb7c41499673d037f1efb998f82"
server_key_iv: "21df3de45b3ac84005f6ae3d1108748b"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.271956 22485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 02:27:53.272845 22500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.273006 22485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.273118 22485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "7526b131e1574fd7a8b4c70f9c0bab3f"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "cb49cdb7c41499673d037f1efb998f82"
server_key_iv: "21df3de45b3ac84005f6ae3d1108748b"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.273190 22485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:53.286455 22485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:53.288260 22485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:53.288419 22485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:53.292234 22485 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:37301
I20260925 02:27:53.292294 22554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:37301 every 8 connection(s)
I20260925 02:27:53.292649 22485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:27:53.293552 22555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:53.295552 22555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f: Bootstrap starting.
I20260925 02:27:53.295877 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 22485
I20260925 02:27:53.295953 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:27:53.296172 12147 external_mini_cluster.cc:1428] Setting key e163e79dee3eb34d17295534d1b3a5a8
I20260925 02:27:53.296175 22555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:53.296535 22555 log.cc:824] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:53.297345 22555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f: No bootstrap required, opened a new log
I20260925 02:27:53.299500 22555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7526b131e1574fd7a8b4c70f9c0bab3f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37301 } }
I20260925 02:27:53.299654 22555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:53.299715 22555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7526b131e1574fd7a8b4c70f9c0bab3f, State: Initialized, Role: FOLLOWER
I20260925 02:27:53.299836 22555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7526b131e1574fd7a8b4c70f9c0bab3f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37301 } }
I20260925 02:27:53.299917 22555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:27:53.299939 22555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:27:53.300012 22555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:53.300586 22555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7526b131e1574fd7a8b4c70f9c0bab3f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37301 } }
I20260925 02:27:53.300689 22555 leader_election.cc:302] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7526b131e1574fd7a8b4c70f9c0bab3f; no voters:
I20260925 02:27:53.300830 22555 leader_election.cc:288] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:27:53.300887 22560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:53.301050 22555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:27:53.301059 22560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [term 1 LEADER]: Becoming Leader. State: Replica: 7526b131e1574fd7a8b4c70f9c0bab3f, State: Running, Role: LEADER
I20260925 02:27:53.301182 22560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7526b131e1574fd7a8b4c70f9c0bab3f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37301 } }
I20260925 02:27:53.301497 22560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7526b131e1574fd7a8b4c70f9c0bab3f. Latest consensus state: current_term: 1 leader_uuid: "7526b131e1574fd7a8b4c70f9c0bab3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7526b131e1574fd7a8b4c70f9c0bab3f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37301 } } }
I20260925 02:27:53.301589 22560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:53.301754 22562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7526b131e1574fd7a8b4c70f9c0bab3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7526b131e1574fd7a8b4c70f9c0bab3f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37301 } } }
I20260925 02:27:53.301822 22569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:27:53.301891 22562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f [sys.catalog]: This master's current role is: LEADER
I20260925 02:27:53.302435 22569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[22471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:53.304157 22569 catalog_manager.cc:1347] Generated new cluster ID: cec44bb6bcfe4c3c9f9c348c971344d3
I20260925 02:27:53.304205 22569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:27:53.307372 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.296672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46042 (local address 127.11.220.254:37301)
0925 02:27:53.296864 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.296871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.296894 (+ 23us) server_negotiation.cc:407] Connection header received
0925 02:27:53.297417 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.297425 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.297486 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.297616 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:53.298175 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.299559 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.300480 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.301076 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.301503 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.304009 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:53.304013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:53.304013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:53.304022 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:53.305798 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:53.306194 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:53.306197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:53.306197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:53.306298 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:53.306497 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:53.306498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:53.306498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:53.306779 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:53.306932 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.307033 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.307182 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
I20260925 02:27:53.308177 22569 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:27:53.308704 22569 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:27:53.314244 22569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f: Generated new TSK 0
I20260925 02:27:53.314379 22569 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:27:53.341075 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37301
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:53.414552 22581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:53.414710 22581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:53.414731 22581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:53.415689 22581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:53.415728 22581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:53.415773 22581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:27:53.417229 22581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:53.417441 22581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:53.417660 22581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:53.422657 22589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:53.422638 22586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:53.422631 22587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:53.422727 22581 server_base.cc:1034] running on GCE node
I20260925 02:27:53.422999 22581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:53.423223 22581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:53.424361 22581 hybrid_clock.cc:648] HybridClock initialized: now 1790303273424346 us; error 32 us; skew 500 ppm
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[22471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303273, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:53.426465 22581 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:27:53.426923 22581 webserver.cc:460] Webserver started at http://127.11.220.193:38803/ using document root <none> and password file <none>
I20260925 02:27:53.427084 22581 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:53.427126 22581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:53.427215 22581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:53.428079 22581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "e04cb62ef8564115bcfe2b8757fa1ba1"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "17442365cb992f80ee00a5b9edf4e856"
server_key_iv: "0667e9abd99afb2de8a03c5348e1dc3b"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.428359 22581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "e04cb62ef8564115bcfe2b8757fa1ba1"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "17442365cb992f80ee00a5b9edf4e856"
server_key_iv: "0667e9abd99afb2de8a03c5348e1dc3b"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.429517 22581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:53.430104 22596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.430315 22581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:53.430393 22581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "e04cb62ef8564115bcfe2b8757fa1ba1"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "17442365cb992f80ee00a5b9edf4e856"
server_key_iv: "0667e9abd99afb2de8a03c5348e1dc3b"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.430454 22581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:53.438562 22581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:53.440111 22581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:53.440249 22581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:53.440495 22581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:53.440770 22581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:53.440800 22581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.440817 22581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:53.440863 22581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.446123 22581 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:36325
I20260925 02:27:53.446199 22711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:36325 every 8 connection(s)
I20260925 02:27:53.446451 22581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[22471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303273, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:53.454669 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.447078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:33355 (local address 127.11.220.254:37301)
0925 02:27:53.447156 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.447158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.447320 (+ 162us) server_negotiation.cc:407] Connection header received
0925 02:27:53.447772 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.447773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.447786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.447846 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:53.448748 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.449275 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.450207 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.450421 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.450516 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.452348 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:53.452350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:53.452350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:53.452357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:53.453453 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:53.453812 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:53.453813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:53.453814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:53.453873 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:53.454097 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:53.454098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:53.454098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:53.454203 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:53.454269 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.454499 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.454575 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:27:53.454794 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.447047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:37301 (local address 127.11.220.193:33355)
0925 02:27:53.447209 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:53.447251 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:53.447675 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:53.447925 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:53.447928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:53.448436 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:53.448659 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.448663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.449374 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.449376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.450107 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.450109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.450527 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.450529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.450640 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.451010 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:53.451017 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:53.452232 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:53.453530 (+ 1298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:53.453537 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:53.453539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:53.453728 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:53.453933 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:53.453933 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:53.453934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:53.454028 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:53.454261 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:53.454268 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:53.454384 (+ 116us) client_negotiation.cc:769] Sending connection context
0925 02:27:53.454478 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 02:27:53.454604 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260925 02:27:53.454797 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 22581
I20260925 02:27:53.454893 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:27:53.455088 12147 external_mini_cluster.cc:1428] Setting key 3d6e094fe1b305aac42a8f93c7dec27c
I20260925 02:27:53.455281 22712 heartbeater.cc:347] Connected to a master server at 127.11.220.254:37301
I20260925 02:27:53.455372 22712 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:53.455528 22712 heartbeater.cc:511] Master 127.11.220.254:37301 requested a full tablet report, sending...
I20260925 02:27:53.455986 22517 ts_manager.cc:194] Registered new tserver with Master: e04cb62ef8564115bcfe2b8757fa1ba1 (127.11.220.193:36325)
I20260925 02:27:53.456615 22517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:33355
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:27:53.476924 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37301
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:53.547753 22717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:53.547904 22717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:53.547922 22717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:53.548985 22717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:53.549019 22717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:53.549064 22717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:27:53.550334 22717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:53.550540 22717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:53.550725 22717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:53.555109 22722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:53.555121 22725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:53.555163 22717 server_base.cc:1034] running on GCE node
W20260925 02:27:53.555121 22723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:53.555563 22717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:53.555732 22717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:53.556856 22717 hybrid_clock.cc:648] HybridClock initialized: now 1790303273556845 us; error 25 us; skew 500 ppm
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[22471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303273, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:53.559087 22717 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:27:53.559456 22717 webserver.cc:460] Webserver started at http://127.11.220.194:39443/ using document root <none> and password file <none>
I20260925 02:27:53.559605 22717 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:53.559639 22717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:53.559712 22717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:53.560621 22717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "fa6766d1df8548549b3e923aed406252"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "60229af25d3ab3c7d610cd93a3b6450a"
server_key_iv: "2ba9018ca36ae130cd5bdb5f639b6a3c"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.560889 22717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "fa6766d1df8548549b3e923aed406252"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "60229af25d3ab3c7d610cd93a3b6450a"
server_key_iv: "2ba9018ca36ae130cd5bdb5f639b6a3c"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.561957 22717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:27:53.562556 22732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.562708 22717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:27:53.562780 22717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "fa6766d1df8548549b3e923aed406252"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "60229af25d3ab3c7d610cd93a3b6450a"
server_key_iv: "2ba9018ca36ae130cd5bdb5f639b6a3c"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.562839 22717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:53.572600 22717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:53.574110 22717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:53.574209 22717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:53.574354 22717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:53.574556 22717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:53.574584 22717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.574600 22717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:53.574611 22717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.579828 22717 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:41675
I20260925 02:27:53.579893 22850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:41675 every 8 connection(s)
I20260925 02:27:53.580181 22717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:27:53.580631 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 22717
I20260925 02:27:53.580736 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:27:53.580960 12147 external_mini_cluster.cc:1428] Setting key 4a08b0d8771099edfc3ae7b9899c6f20
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[22471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303273, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
I20260925 02:27:53.589031 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.580873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:46993 (local address 127.11.220.254:37301)
0925 02:27:53.580938 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.580941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.581127 (+ 186us) server_negotiation.cc:407] Connection header received
0925 02:27:53.581587 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.581588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.581604 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.581653 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:53.582491 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.583278 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.584133 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.584340 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.584411 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.586350 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:53.586351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:53.586352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:53.586360 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:53.587680 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:53.588063 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:53.588064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:53.588064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:53.588122 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:53.588371 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:53.588372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:53.588373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:53.588532 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:53.588610 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.588871 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.588944 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:27:53.589200 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.580861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:37301 (local address 127.11.220.194:46993)
0925 02:27:53.581017 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:53.581051 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:53.581492 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:53.581710 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:53.581713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:53.582232 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:53.582395 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.582399 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.583381 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.583382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.584059 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.584061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.584431 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.584432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.584562 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.584993 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:53.585001 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:53.586260 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:53.587771 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:53.587778 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:53.587780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:53.587981 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:53.588186 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:53.588186 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:53.588187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:53.588288 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:53.588622 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:53.588629 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:53.588748 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 02:27:53.588847 (+ 99us) client_negotiation.cc:240] Negotiation successful
0925 02:27:53.588985 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":97,"threads_started":1}
I20260925 02:27:53.589612 22851 heartbeater.cc:347] Connected to a master server at 127.11.220.254:37301
I20260925 02:27:53.589691 22851 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:53.589833 22851 heartbeater.cc:511] Master 127.11.220.254:37301 requested a full tablet report, sending...
I20260925 02:27:53.590191 22517 ts_manager.cc:194] Registered new tserver with Master: fa6766d1df8548549b3e923aed406252 (127.11.220.194:41675)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
I20260925 02:27:53.590703 22517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:46993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:27:53.602428 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37301
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:27:53.675868 22856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:27:53.676070 22856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:27:53.676105 22856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:27:53.677268 22856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:27:53.677323 22856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:27:53.677364 22856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:27:53.678658 22856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:27:53.678907 22856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:27:53.679105 22856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:27:53.683743 22864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:53.683780 22862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:27:53.683784 22861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:27:53.683809 22856 server_base.cc:1034] running on GCE node
I20260925 02:27:53.684034 22856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:27:53.684305 22856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:27:53.685437 22856 hybrid_clock.cc:648] HybridClock initialized: now 1790303273685424 us; error 38 us; skew 500 ppm
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[22471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303273, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:27:53.687654 22856 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:27:53.688026 22856 webserver.cc:460] Webserver started at http://127.11.220.195:44383/ using document root <none> and password file <none>
I20260925 02:27:53.688259 22856 fs_manager.cc:359] Metadata directory not provided
I20260925 02:27:53.688323 22856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:27:53.688503 22856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:27:53.689445 22856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "8d477676f3b5dfd561dfc10ce860d2af"
server_key_iv: "8d1ca2a9085d5154c90258188ecd8853"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.689770 22856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "8d477676f3b5dfd561dfc10ce860d2af"
server_key_iv: "8d1ca2a9085d5154c90258188ecd8853"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.690989 22856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:27:53.691663 22871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.691829 22856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:27:53.691946 22856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18"
format_stamp: "Formatted at 2026-09-25 02:27:53 on dist-test-slave-dn4n"
server_key: "8d477676f3b5dfd561dfc10ce860d2af"
server_key_iv: "8d1ca2a9085d5154c90258188ecd8853"
server_key_version: "encryptionkey@0"
I20260925 02:27:53.692021 22856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:27:53.710476 22856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:27:53.711961 22856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:27:53.712096 22856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:27:53.712277 22856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:27:53.712685 22856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:27:53.712759 22856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.712797 22856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:27:53.712842 22856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:27:53.718156 22856 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:40307
I20260925 02:27:53.718254 22988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:40307 every 8 connection(s)
I20260925 02:27:53.718497 22856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[22471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303273, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:27:53.726171 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 22856
I20260925 02:27:53.726274 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:27:53.726472 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.719101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:44741 (local address 127.11.220.254:37301)
0925 02:27:53.719144 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.719147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.719322 (+ 175us) server_negotiation.cc:407] Connection header received
0925 02:27:53.719709 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.719710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.719719 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.719756 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:53.720796 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.721418 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.722301 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.722464 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.722507 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.724239 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:53.724241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:53.724241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:53.724247 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:53.725293 (+ 1046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:53.725601 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:53.725602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:53.725602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:53.725654 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:53.725844 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:53.725845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:53.725845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:53.725932 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:53.725977 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.726346 (+ 369us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.726415 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:27:53.726475 12147 external_mini_cluster.cc:1428] Setting key a76d5c5cd99ff5ff4bf5eb26c24af885
I20260925 02:27:53.726657 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.719082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:37301 (local address 127.11.220.195:44741)
0925 02:27:53.719215 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:53.719244 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:53.719627 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:53.719801 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:53.719804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:53.720425 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:27:53.720706 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.720713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.721501 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.721502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.722222 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.722224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.722527 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.722528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.722612 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.722979 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:27:53.722985 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:27:53.724165 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:27:53.725352 (+ 1187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:53.725357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:53.725359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:53.725541 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:53.725712 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:27:53.725712 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:27:53.725713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:27:53.725795 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:27:53.726004 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:27:53.726021 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:27:53.726181 (+ 160us) client_negotiation.cc:769] Sending connection context
0925 02:27:53.726322 (+ 141us) client_negotiation.cc:240] Negotiation successful
0925 02:27:53.726467 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260925 02:27:53.727154 22989 heartbeater.cc:347] Connected to a master server at 127.11.220.254:37301
I20260925 02:27:53.727236 22989 heartbeater.cc:464] Registering TS with master...
I20260925 02:27:53.727378 22989 heartbeater.cc:511] Master 127.11.220.254:37301 requested a full tablet report, sending...
I20260925 02:27:53.727690 22517 ts_manager.cc:194] Registered new tserver with Master: b4f8e0e7dcb045e5ae9e24975bd23d18 (127.11.220.195:40307)
I20260925 02:27:53.728057 22517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:44741
I20260925 02:27:53.737943 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:27:53.744223 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.739151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46052 (local address 127.11.220.254:37301)
0925 02:27:53.739210 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.739212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.739292 (+ 80us) server_negotiation.cc:407] Connection header received
0925 02:27:53.739328 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.739329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.739337 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.739380 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:27:53.739894 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.740350 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.741024 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.741184 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.741230 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.742169 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:27:53.742170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:27:53.742170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:27:53.742175 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:27:53.743386 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:53.743633 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:53.743633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:53.743634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:53.743676 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:27:53.743800 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:27:53.743800 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:27:53.743801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:27:53.743901 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:27:53.743963 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.744042 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.744095 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:27:53.745486 22517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:27:53.746928 22517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:27:53.752833 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.748890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40307 (local address 127.0.0.1:57584)
0925 02:27:53.749133 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:53.749141 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:53.749179 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:53.749538 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:53.749539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:53.749541 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:53.749587 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.749588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.750797 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.750798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.751756 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.751758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.752557 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.752558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.752629 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.752641 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 02:27:53.752710 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:27:53.752754 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":51,"threads_started":1}
I20260925 02:27:53.752839 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.749232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57584 (local address 127.11.220.195:40307)
0925 02:27:53.749372 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.749376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.749383 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:53.749420 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.749421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.749430 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.749519 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:53.749648 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.750679 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.751840 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.752442 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.752550 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.752599 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.752733 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.752768 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260925 02:27:53.753209 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.748887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:36325 (local address 127.0.0.1:50440)
0925 02:27:53.749036 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:53.749069 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:53.749138 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:53.749898 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:53.749899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:53.749901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:53.749936 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.749937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.750855 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.750856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.752116 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.752118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.752984 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.752985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.753049 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.753052 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:53.753104 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 02:27:53.753140 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":97,"thread_start_us":86,"threads_started":1}
I20260925 02:27:53.753221 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.749041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50440 (local address 127.11.220.193:36325)
0925 02:27:53.749604 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.749608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.749616 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:53.749723 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.749726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.749801 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.749898 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:53.750018 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.750707 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.752208 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.752916 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.752985 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.753032 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.753116 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.753152 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":99,"threads_started":1}
I20260925 02:27:53.753496 22921 tablet_service.cc:1467] Processing CreateTablet for tablet 1ff536cebc2843afa0cd3c49f07067bf (DEFAULT_TABLE table=test-table [id=3aa8e96d0bc54cc9a57cfe62e5837261]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:53.753595 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.748911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:41675 (local address 127.0.0.1:46552)
0925 02:27:53.749235 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:53.749242 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:53.749271 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:53.749488 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:53.749490 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:53.749492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:53.749548 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.749550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.750829 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.750830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.752112 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.752114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.753179 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.753180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.753235 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.753239 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:27:53.753500 (+ 261us) client_negotiation.cc:240] Negotiation successful
0925 02:27:53.753541 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":57,"thread_start_us":78,"threads_started":1}
I20260925 02:27:53.753749 22646 tablet_service.cc:1467] Processing CreateTablet for tablet 1ff536cebc2843afa0cd3c49f07067bf (DEFAULT_TABLE table=test-table [id=3aa8e96d0bc54cc9a57cfe62e5837261]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:53.753811 22921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ff536cebc2843afa0cd3c49f07067bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:53.753960 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.749141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.11.220.194:41675)
0925 02:27:53.749327 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.749331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.749340 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:27:53.749374 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.749376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.749390 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.749459 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:53.749681 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.750715 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.752203 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.752792 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.752850 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.753146 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.753793 (+ 647us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.753838 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260925 02:27:53.754035 22646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ff536cebc2843afa0cd3c49f07067bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:53.754750 22781 tablet_service.cc:1467] Processing CreateTablet for tablet 1ff536cebc2843afa0cd3c49f07067bf (DEFAULT_TABLE table=test-table [id=3aa8e96d0bc54cc9a57cfe62e5837261]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:27:53.755041 22781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ff536cebc2843afa0cd3c49f07067bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:27:53.756428 23003 tablet_bootstrap.cc:492] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1: Bootstrap starting.
I20260925 02:27:53.756932 23004 tablet_bootstrap.cc:492] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: Bootstrap starting.
I20260925 02:27:53.757161 23003 tablet_bootstrap.cc:654] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:53.757525 23004 tablet_bootstrap.cc:654] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:53.757537 23003 log.cc:824] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:53.758015 23004 log.cc:824] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:53.758188 23006 tablet_bootstrap.cc:492] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: Bootstrap starting.
I20260925 02:27:53.758360 23003 tablet_bootstrap.cc:492] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1: No bootstrap required, opened a new log
I20260925 02:27:53.758432 23003 ts_tablet_manager.cc:1397] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:27:53.758811 23006 tablet_bootstrap.cc:654] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: Neither blocks nor log segments found. Creating new log.
I20260925 02:27:53.759351 23006 log.cc:824] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: Log is configured to *not* fsync() on all Append() calls
I20260925 02:27:53.760066 23006 tablet_bootstrap.cc:492] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: No bootstrap required, opened a new log
I20260925 02:27:53.760151 23006 ts_tablet_manager.cc:1397] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:27:53.760485 23003 raft_consensus.cc:348] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 } } peers { permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 } } peers { permanent_uuid: "fa6766d1df8548549b3e923aed406252" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 41675 } }
I20260925 02:27:53.760636 23003 raft_consensus.cc:374] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:53.760684 23003 raft_consensus.cc:729] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04cb62ef8564115bcfe2b8757fa1ba1, State: Initialized, Role: FOLLOWER
I20260925 02:27:53.760818 23003 consensus_queue.cc:260] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 } } peers { permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 } } peers { permanent_uuid: "fa6766d1df8548549b3e923aed406252" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 41675 } }
I20260925 02:27:53.761014 23004 tablet_bootstrap.cc:492] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: No bootstrap required, opened a new log
I20260925 02:27:53.761065 23003 ts_tablet_manager.cc:1428] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:27:53.761096 23004 ts_tablet_manager.cc:1397] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 02:27:53.761200 22712 heartbeater.cc:503] Master 127.11.220.254:37301 was elected leader, sending a full tablet report...
I20260925 02:27:53.761874 23006 raft_consensus.cc:348] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 } } peers { permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 } } peers { permanent_uuid: "fa6766d1df8548549b3e923aed406252" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 41675 } }
I20260925 02:27:53.761977 23006 raft_consensus.cc:374] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:53.761996 23006 raft_consensus.cc:729] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa6766d1df8548549b3e923aed406252, State: Initialized, Role: FOLLOWER
I20260925 02:27:53.762090 23006 consensus_queue.cc:260] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 } } peers { permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 } } peers { permanent_uuid: "fa6766d1df8548549b3e923aed406252" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 41675 } }
I20260925 02:27:53.762331 23006 ts_tablet_manager.cc:1428] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:27:53.762439 22851 heartbeater.cc:503] Master 127.11.220.254:37301 was elected leader, sending a full tablet report...
I20260925 02:27:53.762861 23004 raft_consensus.cc:348] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 } } peers { permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 } } peers { permanent_uuid: "fa6766d1df8548549b3e923aed406252" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 41675 } }
I20260925 02:27:53.763013 23004 raft_consensus.cc:374] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:27:53.763058 23004 raft_consensus.cc:729] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f8e0e7dcb045e5ae9e24975bd23d18, State: Initialized, Role: FOLLOWER
I20260925 02:27:53.763195 23004 consensus_queue.cc:260] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 } } peers { permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 } } peers { permanent_uuid: "fa6766d1df8548549b3e923aed406252" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 41675 } }
I20260925 02:27:53.763415 23004 ts_tablet_manager.cc:1428] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:27:53.763469 22989 heartbeater.cc:503] Master 127.11.220.254:37301 was elected leader, sending a full tablet report...
W20260925 02:27:53.830691 22852 tablet.cc:2369] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:27:53.869648 23010 raft_consensus.cc:482] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:27:53.869731 23010 raft_consensus.cc:504] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 } } peers { permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 } } peers { permanent_uuid: "fa6766d1df8548549b3e923aed406252" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 41675 } }
I20260925 02:27:53.870116 23010 leader_election.cc:288] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e04cb62ef8564115bcfe2b8757fa1ba1 (127.11.220.193:36325), b4f8e0e7dcb045e5ae9e24975bd23d18 (127.11.220.195:40307)
I20260925 02:27:53.872822 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.870295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:42749 (local address 127.11.220.193:36325)
0925 02:27:53.870333 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.870334 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.870342 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:27:53.870414 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.870415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.870425 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.870490 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:53.870635 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.871115 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.872017 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.872400 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.872553 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.872563 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.872725 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.872761 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:27:53.872824 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.870228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:36325 (local address 127.11.220.194:42749)
0925 02:27:53.870276 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:53.870287 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:53.870330 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:53.870495 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:53.870496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:53.870498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:53.870567 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.870568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.871208 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.871209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.871927 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.871929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.872552 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.872553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.872632 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.872635 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:27:53.872718 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 02:27:53.872756 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":2}
I20260925 02:27:53.872948 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.870369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:37197 (local address 127.11.220.195:40307)
0925 02:27:53.870420 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.870422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.870449 (+ 27us) server_negotiation.cc:407] Connection header received
0925 02:27:53.870527 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.870528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.870535 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.870615 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:27:53.870766 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.871250 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.872157 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.872621 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.872704 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.872711 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.872853 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.872888 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 02:27:53.872961 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.870248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40307 (local address 127.11.220.194:37197)
0925 02:27:53.870396 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:53.870406 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:27:53.870459 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:27:53.870623 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:27:53.870623 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:27:53.870625 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:27:53.870700 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.870702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.871336 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.871337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.872064 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:27:53.872066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.872704 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.872705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:27:53.872780 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.872782 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:27:53.872848 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 02:27:53.872884 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260925 02:27:53.873202 22666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ff536cebc2843afa0cd3c49f07067bf" candidate_uuid: "fa6766d1df8548549b3e923aed406252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" is_pre_election: true
I20260925 02:27:53.873317 22666 raft_consensus.cc:2454] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa6766d1df8548549b3e923aed406252 in term 0.
I20260925 02:27:53.873515 22736 leader_election.cc:302] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e04cb62ef8564115bcfe2b8757fa1ba1, fa6766d1df8548549b3e923aed406252; no voters:
I20260925 02:27:53.873574 22942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ff536cebc2843afa0cd3c49f07067bf" candidate_uuid: "fa6766d1df8548549b3e923aed406252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" is_pre_election: true
I20260925 02:27:53.873709 23010 raft_consensus.cc:2781] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:27:53.873749 22942 raft_consensus.cc:2454] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa6766d1df8548549b3e923aed406252 in term 0.
I20260925 02:27:53.873770 23010 raft_consensus.cc:482] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:27:53.873798 23010 raft_consensus.cc:3037] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:53.874548 23010 raft_consensus.cc:504] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 } } peers { permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 } } peers { permanent_uuid: "fa6766d1df8548549b3e923aed406252" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 41675 } }
I20260925 02:27:53.874825 23010 leader_election.cc:288] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [CANDIDATE]: Term 1 election: Requested vote from peers e04cb62ef8564115bcfe2b8757fa1ba1 (127.11.220.193:36325), b4f8e0e7dcb045e5ae9e24975bd23d18 (127.11.220.195:40307)
I20260925 02:27:53.874822 22666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ff536cebc2843afa0cd3c49f07067bf" candidate_uuid: "fa6766d1df8548549b3e923aed406252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1"
I20260925 02:27:53.874889 22666 raft_consensus.cc:3037] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:53.874847 22942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ff536cebc2843afa0cd3c49f07067bf" candidate_uuid: "fa6766d1df8548549b3e923aed406252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18"
I20260925 02:27:53.874959 22942 raft_consensus.cc:3037] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:27:53.875586 22666 raft_consensus.cc:2454] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa6766d1df8548549b3e923aed406252 in term 1.
I20260925 02:27:53.875653 22942 raft_consensus.cc:2454] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa6766d1df8548549b3e923aed406252 in term 1.
I20260925 02:27:53.875839 22733 leader_election.cc:302] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f8e0e7dcb045e5ae9e24975bd23d18, fa6766d1df8548549b3e923aed406252; no voters:
I20260925 02:27:53.875972 23010 raft_consensus.cc:2781] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:27:53.876138 23010 raft_consensus.cc:686] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 1 LEADER]: Becoming Leader. State: Replica: fa6766d1df8548549b3e923aed406252, State: Running, Role: LEADER
I20260925 02:27:53.876253 23010 consensus_queue.cc:237] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 } } peers { permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 } } peers { permanent_uuid: "fa6766d1df8548549b3e923aed406252" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 41675 } }
I20260925 02:27:53.877029 22517 catalog_manager.cc:5495] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa6766d1df8548549b3e923aed406252 (127.11.220.194). New cstate: current_term: 1 leader_uuid: "fa6766d1df8548549b3e923aed406252" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa6766d1df8548549b3e923aed406252" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 41675 } health_report { overall_health: HEALTHY } } }
I20260925 02:27:53.895615 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.893408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46558 (local address 127.11.220.194:41675)
0925 02:27:53.893450 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:27:53.893452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:27:53.893459 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:27:53.893501 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:27:53.893501 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:27:53.893515 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:27:53.893564 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:27:53.893680 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.894112 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.894885 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:27:53.895119 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:27:53.895171 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:27:53.895295 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:27:53.895376 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:27:53.895430 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 02:27:53.895485 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 02:27:53.895501 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:27:53.897192 22666 raft_consensus.cc:1261] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1 [term 1 FOLLOWER]: Refusing update from remote peer fa6766d1df8548549b3e923aed406252: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:53.897210 22942 raft_consensus.cc:1261] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18 [term 1 FOLLOWER]: Refusing update from remote peer fa6766d1df8548549b3e923aed406252: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:27:53.897511 23013 consensus_queue.cc:1035] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e04cb62ef8564115bcfe2b8757fa1ba1" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:53.897646 23010 consensus_queue.cc:1035] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f8e0e7dcb045e5ae9e24975bd23d18" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:27:53.899798 23016 mvcc.cc:205] Tried to move back new op lower bound from 7333082209880559616 to 7333082209797689344. Current Snapshot: MvccSnapshot[applied={T|T < 7333082209880559616}]
I20260925 02:27:53.900014 23018 mvcc.cc:205] Tried to move back new op lower bound from 7333082209880559616 to 7333082209797689344. Current Snapshot: MvccSnapshot[applied={T|T < 7333082209880559616}]
I20260925 02:27:53.900166 23017 mvcc.cc:205] Tried to move back new op lower bound from 7333082209880559616 to 7333082209797689344. Current Snapshot: MvccSnapshot[applied={T|T < 7333082209880559616}]
I20260925 02:27:53.902400 22517 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:53.902483 22517 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:27:53.903211 22517 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7526b131e1574fd7a8b4c70f9c0bab3f: Sending DeleteTablet for 3 replicas of tablet 1ff536cebc2843afa0cd3c49f07067bf
I20260925 02:27:53.903508 22646 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ff536cebc2843afa0cd3c49f07067bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:53 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:50440
I20260925 02:27:53.903575 22781 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ff536cebc2843afa0cd3c49f07067bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:53 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:46552
I20260925 02:27:53.903717 23026 tablet_replica.cc:330] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1: stopping tablet replica
I20260925 02:27:53.903786 23027 tablet_replica.cc:330] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: stopping tablet replica
I20260925 02:27:53.903813 23026 raft_consensus.cc:2229] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:53.903858 23027 raft_consensus.cc:2229] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:27:53.903833 22921 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ff536cebc2843afa0cd3c49f07067bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:27:53 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:57584
I20260925 02:27:53.903903 23026 raft_consensus.cc:2258] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:53.903955 23027 raft_consensus.cc:2258] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:53.904333 23026 ts_tablet_manager.cc:1905] T 1ff536cebc2843afa0cd3c49f07067bf P e04cb62ef8564115bcfe2b8757fa1ba1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:53.904453 23027 ts_tablet_manager.cc:1905] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:27:53.904685 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 22581
I20260925 02:27:53.907066 23027 ts_tablet_manager.cc:1918] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:53.907124 23027 log.cc:1196] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/1ff536cebc2843afa0cd3c49f07067bf
I20260925 02:27:53.907347 23027 ts_tablet_manager.cc:1939] T 1ff536cebc2843afa0cd3c49f07067bf P fa6766d1df8548549b3e923aed406252: Deleting consensus metadata
I20260925 02:27:53.907648 22503 catalog_manager.cc:4841] TS fa6766d1df8548549b3e923aed406252 (127.11.220.194:41675): tablet 1ff536cebc2843afa0cd3c49f07067bf (table test-table [id=3aa8e96d0bc54cc9a57cfe62e5837261]) successfully deleted
I20260925 02:27:53.909332 23028 tablet_replica.cc:330] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: stopping tablet replica
I20260925 02:27:53.909446 23028 raft_consensus.cc:2229] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:27:53.909564 23028 raft_consensus.cc:2258] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:27:53.909929 23028 ts_tablet_manager.cc:1905] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 02:27:53.910403 22504 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:36325 (error 108)
I20260925 02:27:53.910889 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:27:53.910756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:36325 (local address 127.0.0.1:50456)
0925 02:27:53.910819 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 02:27:53.910861 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:36325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 02:27:53.911396 22504 catalog_manager.cc:4584] TS e04cb62ef8564115bcfe2b8757fa1ba1 (127.11.220.193:36325): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ff536cebc2843afa0cd3c49f07067bf: Network error: Client connection negotiation failed: client connection to 127.11.220.193:36325: connect: Connection refused (error 111)
I20260925 02:27:53.911559 23028 ts_tablet_manager.cc:1918] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:27:53.911564 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 22717
I20260925 02:27:53.911639 23028 log.cc:1196] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/1ff536cebc2843afa0cd3c49f07067bf
I20260925 02:27:53.911867 23028 ts_tablet_manager.cc:1939] T 1ff536cebc2843afa0cd3c49f07067bf P b4f8e0e7dcb045e5ae9e24975bd23d18: Deleting consensus metadata
I20260925 02:27:53.912470 22501 catalog_manager.cc:4841] TS b4f8e0e7dcb045e5ae9e24975bd23d18 (127.11.220.195:40307): tablet 1ff536cebc2843afa0cd3c49f07067bf (table test-table [id=3aa8e96d0bc54cc9a57cfe62e5837261]) successfully deleted
I20260925 02:27:53.916172 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 22856
I20260925 02:27:53.922601 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 22485
2026-09-25T02:27:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1792 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[23032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[23032](info): set up 2 sockets
Sep 25 02:27:53 dist-test-slave-dn4n krb5kdc[23032](info): commencing operation
krb5kdc: starting...
W20260925 02:27:54.878041 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:27:54 dist-test-slave-dn4n krb5kdc[23032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:27:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:27:54Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.220.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.220.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/rangeradmin_127.11.220.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/rangeradmin_127.11.220.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.220.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.220.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/rangerlookup_127.11.220.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/rangerlookup_127.11.220.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.220.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.220.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/HTTP_127.11.220.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/HTTP_127.11.220.212@KRBTEST.COM.keytab.
I20260925 02:27:55.026902 12147 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfV0OY1/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/postgres -l logfile start
2026-09-25 02:27:57.254 UTC [23062] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 02:27:57.254 UTC [23062] LOG: listening on IPv4 address "127.11.220.212", port 40759
2026-09-25 02:27:57.258 UTC [23062] LOG: listening on Unix socket "/tmp/.s.PGSQL.40759"
2026-09-25 02:27:57.290 UTC [23065] LOG: database system was shut down at 2026-09-25 02:27:56 UTC
2026-09-25 02:27:57.294 UTC [23062] LOG: database system is ready to accept connections
I20260925 02:27:58.157835 12147 mini_postgres.cc:96] Postgres bound to 40759
2026-09-25 02:27:58.164 UTC [23073] FATAL: database "slave" does not exist
127.11.220.212:40759 - accepting connections
I20260925 02:27:58.164921 12147 mini_ranger.cc:161] Starting Ranger...
I20260925 02:27:58.178912 12147 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 02:27:58.565258 12147 mini_ranger.cc:87] Created ranger Postgres database
I20260925 02:27:58.565339 12147 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-admin
2026-09-25 02:27:58,915 [I] DB FLAVOR :POSTGRES
2026-09-25 02:27:58,915 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 02:27:58,916 [I] Checking connection..
2026-09-25 02:27:58,916 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 02:27:59,153 [I] Checking connection passed.
2026-09-25 02:27:59,153 [I] --------- Verifying version history table ---------
2026-09-25 02:27:59,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:27:59,383 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 02:27:59,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:27:59,585 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 02:27:59,586 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 02:27:59,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 02:27:59.794 UTC [23140] WARNING: there is no transaction in progress
2026-09-25 02:27:59,800 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 02:27:59,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:28:00,001 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 02:28:00,001 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 02:28:00,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 02:28:00,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 02:28:00,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-dn4n.c.gcp-upstream.internal','N') ;"
2026-09-25 02:28:00,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 02:28:00,885 [I] Table x_portal_user does not exist in database ranger
2026-09-25 02:28:00,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 02:28:01,089 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 02:28:01,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 02:28:01,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 02:28:01,490 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 02:28:01,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 02:28:02.286 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.300 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.314 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.327 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.340 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.355 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.403 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.409 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.422 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.431 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.440 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.452 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.460 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.465 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.471 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02.476 UTC [23251] WARNING: there is no transaction in progress
2026-09-25 02:28:02,995 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 02:28:02,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 02:28:03,216 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 02:28:03,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 02:28:03,439 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 02:28:03,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 02:28:03,640 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 02:28:03,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 02:28:03,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 02:28:04,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-dn4n.c.gcp-upstream.internal';"
2026-09-25 02:28:04,306 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 02:28:04,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 02:28:04,515 [I] DB_PATCHES have already been applied
I20260925 02:28:04.520952 12147 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfV0OY1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 02:28:04.521006 12147 mini_ranger.cc:193] Using host: 127.11.220.212
I20260925 02:28:04.524165 12147 mini_ranger.cc:227] Ranger admin URL: http://127.11.220.212:35333
Sep 25, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 2:28:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 2:28:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 2:28:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 02:28:05.540073 12147 mini_ranger.cc:160] Time spent starting Ranger: real 7.375s user 0.000s sys 0.008s
Sep 25, 2026 2:28:24 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 2:28:24 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 25, 2026 2:28:24 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 2:28:24 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 2:28:25 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 02:28:25.697036 12147 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.220.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.220.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/rangerkms_127.11.220.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/rangerkms_127.11.220.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.220.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.220.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.220.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/HTTP_127.11.220.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.220.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/HTTP_127.11.220.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 02:28:25 dist-test-slave-dn4n krb5kdc[23032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303305, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 02:28:25.768581 12147 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 02:28:25.776549 12147 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 02:28:26.500794 12147 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 02:28:26.500877 12147 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-kms
I20260925 02:28:26.500900 12147 mini_ranger_kms.cc:227] Using postgres at 127.11.220.212:40759
2026-09-25 02:28:27,015 [I] DB FLAVOR :POSTGRES
2026-09-25 02:28:27,015 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 02:28:27,015 [I] Checking connection
2026-09-25 02:28:27,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 02:28:27,228 [I] connection success
2026-09-25 02:28:27,228 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 02:28:27,229 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 02:28:27,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 02:28:27,455 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 02:28:27,455 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 02:28:27,455 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 02:28:27,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:40759/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 02:28:27,677 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 02:28:28.076967 12147 mini_ranger_kms.cc:310] Created kms service
I20260925 02:28:28.258778 12147 mini_ranger_kms.cc:326] Created kudu user
I20260925 02:28:28.298077 12147 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 02:28:28 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303305, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:28:28.659325 12147 mini_ranger_kms.cc:384] Added ranger policy
I20260925 02:28:28.659433 12147 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 02:28:28.659456 12147 mini_ranger_kms.cc:240] Using host: 127.11.220.212
I20260925 02:28:28.662808 12147 mini_ranger_kms.cc:276] Ranger KMS PID: 23436
I20260925 02:28:28.662923 12147 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.220.212:45071
Sep 25, 2026 2:28:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:28:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:28:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 2:28:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 2:28:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 2:28:29 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 25 02:28:30 dist-test-slave-dn4n krb5kdc[23032](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790303310, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.220.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:28:30.740865 12147 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.972s user 0.000s sys 0.207s
I20260925 02:28:30.740978 12147 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 02:28:30.741067 12147 mini_ranger_kms.cc:401] 127.11.220.212:45071/kms/v1/keys
Sep 25 02:28:31 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303310, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.220.212@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
Sep 25 02:28:31 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303310, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.220.212@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
Sep 25 02:28:33 dist-test-slave-dn4n krb5kdc[23032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:28:33.430761 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:41879
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:41879
--ranger_config_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:45071/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:28:33.517225 23489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:28:33.517390 23489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:28:33.517408 23489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:28:33.518417 23489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:28:33.518453 23489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:28:33.518466 23489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:28:33.518476 23489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:28:33.518487 23489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:28:33.519878 23489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.220.212:45071/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:41879
--ranger_config_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:41879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:28:33.520349 23489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:28:33.520637 23489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:28:33.527055 23495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:28:33.527123 23494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:28:33.527268 23497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:28:33.527133 23489 server_base.cc:1034] running on GCE node
I20260925 02:28:33.527495 23489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:28:33.527762 23489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:28:33.528949 23489 hybrid_clock.cc:648] HybridClock initialized: now 1790303313528935 us; error 29 us; skew 500 ppm
Sep 25 02:28:33 dist-test-slave-dn4n krb5kdc[23032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303313, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:28:33.531307 23489 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:28:33.531713 23489 webserver.cc:460] Webserver started at http://127.11.220.254:44737/ using document root <none> and password file <none>
I20260925 02:28:33.531906 23489 fs_manager.cc:359] Metadata directory not provided
I20260925 02:28:33.531963 23489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:28:33.532109 23489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:28:33 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303313, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:28:33.589068 23489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0"
format_stamp: "Formatted at 2026-09-25 02:28:33 on dist-test-slave-dn4n"
server_key: "59e44daf9a12c61e992fe121dc5378d9"
server_key_iv: "576b5e5edbca0c1f348b6f46a979b18a"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:33.589470 23489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0"
format_stamp: "Formatted at 2026-09-25 02:28:33 on dist-test-slave-dn4n"
server_key: "59e44daf9a12c61e992fe121dc5378d9"
server_key_iv: "576b5e5edbca0c1f348b6f46a979b18a"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:33.590879 23489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:28:33.603720 23504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:28:33.603929 23489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:28:33.604022 23489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0"
format_stamp: "Formatted at 2026-09-25 02:28:33 on dist-test-slave-dn4n"
server_key: "59e44daf9a12c61e992fe121dc5378d9"
server_key_iv: "576b5e5edbca0c1f348b6f46a979b18a"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:33.604100 23489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:28:33.618178 23489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:28:33.619985 23489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:28:33.620182 23489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:28:33.625012 23489 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:41879
I20260925 02:28:33.625032 23556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:41879 every 8 connection(s)
I20260925 02:28:33.625325 23489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:28:33.625927 23557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:28:33.626085 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 23489
I20260925 02:28:33.626179 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:28:33.626366 12147 external_mini_cluster.cc:1428] Setting key 73ce6785b038ec34b305cb0bf67952f3
I20260925 02:28:33.629027 23557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0: Bootstrap starting.
I20260925 02:28:33.629554 23557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0: Neither blocks nor log segments found. Creating new log.
I20260925 02:28:33.629902 23557 log.cc:824] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0: Log is configured to *not* fsync() on all Append() calls
I20260925 02:28:33.630643 23557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0: No bootstrap required, opened a new log
I20260925 02:28:33.633355 23557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 41879 } }
I20260925 02:28:33.633507 23557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:28:33.633566 23557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b78c07224a346cb8da9bc0d7f7b3eb0, State: Initialized, Role: FOLLOWER
Sep 25 02:28:33 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:28:33.633709 23557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 41879 } }
I20260925 02:28:33.633808 23557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:28:33.633862 23557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:28:33.633913 23557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:28:33.634573 23557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 41879 } }
I20260925 02:28:33.634708 23557 leader_election.cc:302] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b78c07224a346cb8da9bc0d7f7b3eb0; no voters:
I20260925 02:28:33.634866 23557 leader_election.cc:288] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:28:33.635128 23562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:28:33.635206 23557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:28:33.635340 23562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [term 1 LEADER]: Becoming Leader. State: Replica: 6b78c07224a346cb8da9bc0d7f7b3eb0, State: Running, Role: LEADER
I20260925 02:28:33.635479 23562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 41879 } }
I20260925 02:28:33.635857 23562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b78c07224a346cb8da9bc0d7f7b3eb0. Latest consensus state: current_term: 1 leader_uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 41879 } } }
I20260925 02:28:33.635994 23562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:28:33.636543 23557 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 02:28:33.638917 23563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b78c07224a346cb8da9bc0d7f7b3eb0" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 41879 } } }
I20260925 02:28:33.639210 23563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:28:33.639230 23572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:28:33.640218 23572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:28:33.641137 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:33.626838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60180 (local address 127.11.220.254:41879)
0925 02:28:33.627006 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:33.627014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:33.627042 (+ 28us) server_negotiation.cc:407] Connection header received
0925 02:28:33.627624 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:33.627633 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:33.627691 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:33.627821 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:28:33.628498 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:33.629686 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:33.630618 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:33.631274 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:33.631396 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:33.634144 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:28:33.634148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:28:33.634148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:28:33.634159 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:28:33.635915 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:28:33.636306 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:28:33.636309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:28:33.636309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:28:33.636388 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:28:33.636636 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:28:33.636637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:28:33.636638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:28:33.636922 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:28:33.639825 (+ 2903us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:33.640069 (+ 244us) server_negotiation.cc:299] Negotiation successful
0925 02:28:33.640647 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
I20260925 02:28:33.648375 23581 mvcc.cc:205] Tried to move back new op lower bound from 7333082372700663808 to 7333082372651495424. Current Snapshot: MvccSnapshot[applied={T|T < 7333082372700663808}]
I20260925 02:28:33.648844 23572 catalog_manager.cc:1347] Generated new cluster ID: 381279a563384f47931f1ea3e62491ca
I20260925 02:28:33.648898 23572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:28:33.660811 23572 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:28:33.661535 23572 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:28:33.672372 23572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0: Generated new TSK 0
I20260925 02:28:33.672608 23572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:28:34.326892 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:45071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:41879
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:28:34.427654 23602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:28:34.427918 23602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:28:34.427964 23602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:28:34.429191 23602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:28:34.429265 23602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:28:34.429308 23602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:28:34.430959 23602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.220.212:45071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:41879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:28:34.431334 23602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:28:34.431607 23602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:28:34.437223 23612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:28:34.437292 23610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:28:34.437512 23609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:28:34.437634 23602 server_base.cc:1034] running on GCE node
I20260925 02:28:34.437816 23602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:28:34.438060 23602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:28:34.439203 23602 hybrid_clock.cc:648] HybridClock initialized: now 1790303314439194 us; error 23 us; skew 500 ppm
Sep 25 02:28:34 dist-test-slave-dn4n krb5kdc[23032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:28:34.441923 23602 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:28:34.442390 23602 webserver.cc:460] Webserver started at http://127.11.220.193:38691/ using document root <none> and password file <none>
I20260925 02:28:34.442620 23602 fs_manager.cc:359] Metadata directory not provided
I20260925 02:28:34.442680 23602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:28:34.442806 23602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:28:34 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:28:34.454952 23602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "3c292d43594e48f8a1cd30d858f1ca9a"
format_stamp: "Formatted at 2026-09-25 02:28:34 on dist-test-slave-dn4n"
server_key: "3f8f7989f732a527df31bd76d4fae7f7"
server_key_iv: "85af9975e0c8708f37882c536fbb17f3"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:34.455444 23602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "3c292d43594e48f8a1cd30d858f1ca9a"
format_stamp: "Formatted at 2026-09-25 02:28:34 on dist-test-slave-dn4n"
server_key: "3f8f7989f732a527df31bd76d4fae7f7"
server_key_iv: "85af9975e0c8708f37882c536fbb17f3"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:34.459204 23602 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260925 02:28:34.472580 23620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:28:34.472756 23602 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260925 02:28:34.472827 23602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "3c292d43594e48f8a1cd30d858f1ca9a"
format_stamp: "Formatted at 2026-09-25 02:28:34 on dist-test-slave-dn4n"
server_key: "3f8f7989f732a527df31bd76d4fae7f7"
server_key_iv: "85af9975e0c8708f37882c536fbb17f3"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:34.472889 23602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:28:34.508899 23602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:28:34.511102 23602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:28:34.511287 23602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:28:34.511529 23602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:28:34.511861 23602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:28:34.511933 23602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:28:34.512015 23602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:28:34.512079 23602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:28:34.518970 23602 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:39961
I20260925 02:28:34.519436 23602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:28:34.521245 23742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:39961 every 8 connection(s)
I20260925 02:28:34.522567 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 23602
I20260925 02:28:34.522679 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:28:34.522881 12147 external_mini_cluster.cc:1428] Setting key 15a553a3dd188f0df51b975cfed0cddd
Sep 25 02:28:34 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:28:34.532510 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:34.522228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:52173 (local address 127.11.220.254:41879)
0925 02:28:34.522370 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:34.522372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:34.522574 (+ 202us) server_negotiation.cc:407] Connection header received
0925 02:28:34.523385 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:34.523387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:34.523399 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:34.523475 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:28:34.524245 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.525043 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.526459 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.526706 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.526766 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:34.529063 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:28:34.529064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:28:34.529064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:28:34.529071 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:28:34.530765 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:28:34.531362 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:28:34.531363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:28:34.531363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:28:34.531447 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:28:34.531733 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:28:34.531734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:28:34.531735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:28:34.531867 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:28:34.531908 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:34.532301 (+ 393us) server_negotiation.cc:299] Negotiation successful
0925 02:28:34.532385 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260925 02:28:34.532961 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:34.522184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:41879 (local address 127.11.220.193:52173)
0925 02:28:34.522449 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0925 02:28:34.522492 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:28:34.523281 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:28:34.523600 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:28:34.523605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:28:34.523829 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:28:34.524073 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:34.524079 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.525163 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.525164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:34.526372 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:34.526375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.526825 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.526825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:34.526956 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:34.527494 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:28:34.527504 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:28:34.528961 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:28:34.530861 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:28:34.530867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:28:34.530869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:28:34.531132 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:28:34.531519 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:28:34.531532 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:28:34.531532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:28:34.531657 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:28:34.531954 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:28:34.531959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:28:34.532145 (+ 186us) client_negotiation.cc:769] Sending connection context
0925 02:28:34.532608 (+ 463us) client_negotiation.cc:240] Negotiation successful
0925 02:28:34.532757 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":76,"threads_started":1}
I20260925 02:28:34.533555 23743 heartbeater.cc:347] Connected to a master server at 127.11.220.254:41879
I20260925 02:28:34.533635 23743 heartbeater.cc:464] Registering TS with master...
I20260925 02:28:34.533799 23743 heartbeater.cc:511] Master 127.11.220.254:41879 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
I20260925 02:28:34.534348 23521 ts_manager.cc:194] Registered new tserver with Master: 3c292d43594e48f8a1cd30d858f1ca9a (127.11.220.193:39961)
I20260925 02:28:34.535210 23521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:52173
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:28:34.555222 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:45071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:41879
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:28:34 dist-test-slave-dn4n krb5kdc[23032](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 02:28:34.638697 23752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:28:34.639010 23752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:28:34.639066 23752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:28:34.640630 23752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:28:34.640700 23752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:28:34.640748 23752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:28:34.642259 23752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.220.212:45071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:41879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:28:34.642519 23752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:28:34.642831 23752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:28:34.647549 23761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:28:34.647782 23752 server_base.cc:1034] running on GCE node
W20260925 02:28:34.647816 23759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:28:34.647631 23757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:28:34.648073 23752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:28:34.648311 23752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:28:34.649461 23752 hybrid_clock.cc:648] HybridClock initialized: now 1790303314649436 us; error 39 us; skew 500 ppm
Sep 25 02:28:34 dist-test-slave-dn4n krb5kdc[23032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:28:34.651895 23752 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:28:34.652313 23752 webserver.cc:460] Webserver started at http://127.11.220.194:33493/ using document root <none> and password file <none>
I20260925 02:28:34.652576 23752 fs_manager.cc:359] Metadata directory not provided
I20260925 02:28:34.652642 23752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:28:34.652787 23752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:28:34 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:28:34.665079 23752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "7cb47b958dc8446a8dc5c3bb349bbf53"
format_stamp: "Formatted at 2026-09-25 02:28:34 on dist-test-slave-dn4n"
server_key: "2c6d2e551cffe7069222f838a0418125"
server_key_iv: "66971368ae0af3b8bb6cbab519011b0c"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:34.665542 23752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "7cb47b958dc8446a8dc5c3bb349bbf53"
format_stamp: "Formatted at 2026-09-25 02:28:34 on dist-test-slave-dn4n"
server_key: "2c6d2e551cffe7069222f838a0418125"
server_key_iv: "66971368ae0af3b8bb6cbab519011b0c"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:34.667388 23752 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 02:28:34.686573 23769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:28:34.686779 23752 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260925 02:28:34.686862 23752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "7cb47b958dc8446a8dc5c3bb349bbf53"
format_stamp: "Formatted at 2026-09-25 02:28:34 on dist-test-slave-dn4n"
server_key: "2c6d2e551cffe7069222f838a0418125"
server_key_iv: "66971368ae0af3b8bb6cbab519011b0c"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:34.686941 23752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:28:34.715335 23752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:28:34.717721 23752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:28:34.717865 23752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:28:34.718120 23752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:28:34.718487 23752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:28:34.718556 23752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:28:34.718588 23752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:28:34.718657 23752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:28:34.725984 23752 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:42369
I20260925 02:28:34.726449 23752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:28:34.727015 23888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:42369 every 8 connection(s)
I20260925 02:28:34.733192 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 23752
I20260925 02:28:34.733291 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:28:34.733479 12147 external_mini_cluster.cc:1428] Setting key 0647047f36d5cd2cb808d2128a6bab0f
Sep 25 02:28:34 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:28:34.737596 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:34.727395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:57807 (local address 127.11.220.254:41879)
0925 02:28:34.727441 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:34.727443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:34.727740 (+ 297us) server_negotiation.cc:407] Connection header received
0925 02:28:34.728560 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:34.728562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:34.728577 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:34.728609 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:28:34.729464 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.730246 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.731672 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.731945 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.731987 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:34.734314 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:28:34.734315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:28:34.734316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:28:34.734323 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:28:34.735906 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:28:34.736462 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:28:34.736463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:28:34.736464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:28:34.736530 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:28:34.736813 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:28:34.736814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:28:34.736814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:28:34.736947 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:28:34.737023 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:34.737393 (+ 370us) server_negotiation.cc:299] Negotiation successful
0925 02:28:34.737481 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:28:34.738093 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:34.727393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:41879 (local address 127.11.220.194:57807)
0925 02:28:34.727615 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 02:28:34.727657 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 02:28:34.728332 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:28:34.728798 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:28:34.728819 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:28:34.729068 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:28:34.729346 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:34.729352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.730366 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.730367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:34.731577 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:34.731580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.732076 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.732076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:34.732206 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:34.732715 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:28:34.732724 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:28:34.734206 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:28:34.736024 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:28:34.736030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:28:34.736032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:28:34.736349 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:28:34.736606 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:28:34.736606 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:28:34.736607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:28:34.736731 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:28:34.737077 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:28:34.737084 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:28:34.737233 (+ 149us) client_negotiation.cc:769] Sending connection context
0925 02:28:34.737716 (+ 483us) client_negotiation.cc:240] Negotiation successful
0925 02:28:34.737868 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
I20260925 02:28:34.738516 23889 heartbeater.cc:347] Connected to a master server at 127.11.220.254:41879
I20260925 02:28:34.738656 23889 heartbeater.cc:464] Registering TS with master...
I20260925 02:28:34.738862 23889 heartbeater.cc:511] Master 127.11.220.254:41879 requested a full tablet report, sending...
I20260925 02:28:34.739250 23521 ts_manager.cc:194] Registered new tserver with Master: 7cb47b958dc8446a8dc5c3bb349bbf53 (127.11.220.194:42369)
I20260925 02:28:34.739724 23521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:57807
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:28:34.769124 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:45071/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:41879
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:28:34.874188 23894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:28:34.874486 23894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:28:34.874564 23894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:28:34.876078 23894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:28:34.876183 23894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:28:34.876255 23894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:28:34.878293 23894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.220.212:45071/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:41879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:28:34.878687 23894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:28:34.878973 23894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:28:34.884980 23903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:28:34.885622 23900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:28:34.885898 23901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:28:34.886118 23894 server_base.cc:1034] running on GCE node
I20260925 02:28:34.886430 23894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:28:34.886639 23894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:28:34.887794 23894 hybrid_clock.cc:648] HybridClock initialized: now 1790303314887775 us; error 29 us; skew 500 ppm
Sep 25 02:28:34 dist-test-slave-dn4n krb5kdc[23032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:28:34.890793 23894 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:28:34.891263 23894 webserver.cc:460] Webserver started at http://127.11.220.195:33001/ using document root <none> and password file <none>
I20260925 02:28:34.891515 23894 fs_manager.cc:359] Metadata directory not provided
I20260925 02:28:34.891575 23894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:28:34.891687 23894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:28:34 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:28:34.910569 23894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "8296db03f8584f6e82eff3454aa90339"
format_stamp: "Formatted at 2026-09-25 02:28:34 on dist-test-slave-dn4n"
server_key: "ee49874fb126cdfa4e85928f7e9dd749"
server_key_iv: "5e4bdf57423d8d2b7323bdb7f431ee34"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:34.910945 23894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "8296db03f8584f6e82eff3454aa90339"
format_stamp: "Formatted at 2026-09-25 02:28:34 on dist-test-slave-dn4n"
server_key: "ee49874fb126cdfa4e85928f7e9dd749"
server_key_iv: "5e4bdf57423d8d2b7323bdb7f431ee34"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:34.916965 23894 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.000s
I20260925 02:28:34.930160 23910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:28:34.930361 23894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260925 02:28:34.930434 23894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "8296db03f8584f6e82eff3454aa90339"
format_stamp: "Formatted at 2026-09-25 02:28:34 on dist-test-slave-dn4n"
server_key: "ee49874fb126cdfa4e85928f7e9dd749"
server_key_iv: "5e4bdf57423d8d2b7323bdb7f431ee34"
server_key_version: "kuduclusterkey@0"
I20260925 02:28:34.930500 23894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:28:34.958774 23894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:28:34.961134 23894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:28:34.961374 23894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:28:34.961606 23894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:28:34.962019 23894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:28:34.962126 23894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:28:34.962201 23894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:28:34.962258 23894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:28:34.968837 23894 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:35133
I20260925 02:28:34.969311 23894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:28:34.970489 24025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:35133 every 8 connection(s)
I20260925 02:28:34.974411 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 23894
I20260925 02:28:34.974525 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:28:34.974738 12147 external_mini_cluster.cc:1428] Setting key c463ad659b0ce7d064afb8a554b7fd63
Sep 25 02:28:34 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:28:34.981087 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:34.970060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:59297 (local address 127.11.220.254:41879)
0925 02:28:34.970110 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:34.970112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:34.971308 (+ 1196us) server_negotiation.cc:407] Connection header received
0925 02:28:34.971955 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:34.971957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:34.971971 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:34.972001 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:28:34.972793 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.973481 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.975167 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.975464 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.975507 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:34.977956 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:28:34.977957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:28:34.977958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:28:34.977965 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:28:34.979454 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:28:34.979919 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:28:34.979920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:28:34.979920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:28:34.979978 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:28:34.980326 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:28:34.980327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:28:34.980327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:28:34.980481 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:28:34.980527 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:34.980902 (+ 375us) server_negotiation.cc:299] Negotiation successful
0925 02:28:34.980983 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 02:28:34.981750 24028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:34.970055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:41879 (local address 127.11.220.195:59297)
0925 02:28:34.971174 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0925 02:28:34.971220 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 02:28:34.971831 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:28:34.972184 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:28:34.972188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:28:34.972401 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:28:34.972684 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:34.972690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.973598 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.973599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:34.975046 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:34.975050 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.975596 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.975597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:34.975751 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:34.976289 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:28:34.976300 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:28:34.977856 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:28:34.979541 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:28:34.979547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:28:34.979550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:28:34.979813 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:28:34.980108 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:28:34.980109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:28:34.980109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:28:34.980241 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:28:34.980576 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:28:34.980583 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:28:34.980735 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 02:28:34.981397 (+ 662us) client_negotiation.cc:240] Negotiation successful
0925 02:28:34.981541 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1079,"thread_start_us":525,"threads_started":1}
I20260925 02:28:34.982434 24026 heartbeater.cc:347] Connected to a master server at 127.11.220.254:41879
I20260925 02:28:34.982577 24026 heartbeater.cc:464] Registering TS with master...
I20260925 02:28:34.982805 24026 heartbeater.cc:511] Master 127.11.220.254:41879 requested a full tablet report, sending...
I20260925 02:28:34.983253 23521 ts_manager.cc:194] Registered new tserver with Master: 8296db03f8584f6e82eff3454aa90339 (127.11.220.195:35133)
I20260925 02:28:34.983753 23521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:59297
I20260925 02:28:34.986377 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:28:34.995086 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:34.987867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60190 (local address 127.11.220.254:41879)
0925 02:28:34.987904 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:34.987905 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:34.988096 (+ 191us) server_negotiation.cc:407] Connection header received
0925 02:28:34.988190 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:34.988191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:34.988199 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:34.988226 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:28:34.988969 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.989607 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.990679 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:34.990980 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:34.991029 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:34.992423 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:28:34.992425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:28:34.992425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:28:34.992431 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:28:34.993889 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:28:34.994273 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:28:34.994275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:28:34.994275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:28:34.994342 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:28:34.994532 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:28:34.994533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:28:34.994533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:28:34.994630 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:28:34.994696 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:34.994903 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 02:28:34.994968 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:28:34.996984 23521 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:28:34.999083 23521 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:28:35.006455 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.001929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:35133 (local address 127.0.0.1:55100)
0925 02:28:35.002064 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:28:35.002098 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:28:35.002153 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:28:35.002852 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:28:35.002855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:28:35.002857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:28:35.002913 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:35.002914 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.004104 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.004105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:35.005330 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:35.005333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.006204 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.006205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:35.006274 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.006287 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 02:28:35.006351 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:28:35.006393 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
I20260925 02:28:35.006724 24040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.001951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:42369 (local address 127.0.0.1:56382)
0925 02:28:35.002264 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0925 02:28:35.002271 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:28:35.002301 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:28:35.002958 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:28:35.002958 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:28:35.002960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:28:35.003019 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:35.003020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.004241 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.004242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:35.005387 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:35.005390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.006502 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.006503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:35.006572 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.006576 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:28:35.006612 (+ 36us) client_negotiation.cc:240] Negotiation successful
0925 02:28:35.006645 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":83,"threads_started":1}
I20260925 02:28:35.007067 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.002372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56382 (local address 127.11.220.194:42369)
0925 02:28:35.002554 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:35.002558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:35.002568 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:28:35.002609 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:35.002611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:35.002874 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:35.003075 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:28:35.003138 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.004148 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.005743 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.006414 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.006850 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.006911 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:35.006933 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 02:28:35.006997 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260925 02:28:35.007258 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.002537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55100 (local address 127.11.220.195:35133)
0925 02:28:35.002687 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:35.002691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:35.002700 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:28:35.002731 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:35.002732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:35.002746 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:35.002987 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:28:35.003040 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.003981 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.005440 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.006110 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.007072 (+ 962us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.007131 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:35.007156 (+ 25us) server_negotiation.cc:299] Negotiation successful
0925 02:28:35.007189 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260925 02:28:35.007762 23802 tablet_service.cc:1467] Processing CreateTablet for tablet be4bfe2b06734ef0a537ce962ca942bd (DEFAULT_TABLE table=test-table [id=8e2269dc50414f22a093b01c1e145a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:28:35.007962 23939 tablet_service.cc:1467] Processing CreateTablet for tablet be4bfe2b06734ef0a537ce962ca942bd (DEFAULT_TABLE table=test-table [id=8e2269dc50414f22a093b01c1e145a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:28:35.008090 23802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4bfe2b06734ef0a537ce962ca942bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:28:35.008253 23939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4bfe2b06734ef0a537ce962ca942bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:28:35.011576 24046 tablet_bootstrap.cc:492] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: Bootstrap starting.
I20260925 02:28:35.012606 24046 tablet_bootstrap.cc:654] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: Neither blocks nor log segments found. Creating new log.
I20260925 02:28:35.012929 24046 log.cc:824] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: Log is configured to *not* fsync() on all Append() calls
I20260925 02:28:35.013263 24047 tablet_bootstrap.cc:492] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: Bootstrap starting.
I20260925 02:28:35.014338 24047 tablet_bootstrap.cc:654] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: Neither blocks nor log segments found. Creating new log.
I20260925 02:28:35.014671 24047 log.cc:824] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: Log is configured to *not* fsync() on all Append() calls
I20260925 02:28:35.015355 24047 tablet_bootstrap.cc:492] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: No bootstrap required, opened a new log
I20260925 02:28:35.015412 24047 ts_tablet_manager.cc:1397] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:28:35.016089 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.002155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:39961 (local address 127.0.0.1:36794)
0925 02:28:35.006510 (+ 4355us) negotiation.cc:107] Waiting for socket to connect
0925 02:28:35.006520 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:28:35.006554 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:28:35.012768 (+ 6214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:28:35.012769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:28:35.012771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:28:35.012800 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:35.012801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.013953 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.013954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:35.014889 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:35.014892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.015876 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.015877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:35.015938 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.015941 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:28:35.015980 (+ 39us) client_negotiation.cc:240] Negotiation successful
0925 02:28:35.016015 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4347,"thread_start_us":81,"threads_started":1}
I20260925 02:28:35.016511 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.002253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36794 (local address 127.11.220.193:39961)
0925 02:28:35.012593 (+ 10340us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:35.012598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:35.012609 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:28:35.012650 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:35.012651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:35.012667 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:35.012868 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:28:35.012945 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.013863 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.015017 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.015734 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.016272 (+ 538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.016335 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:35.016363 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 02:28:35.016401 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10305,"thread_start_us":50,"threads_started":1}
I20260925 02:28:35.017016 23671 tablet_service.cc:1467] Processing CreateTablet for tablet be4bfe2b06734ef0a537ce962ca942bd (DEFAULT_TABLE table=test-table [id=8e2269dc50414f22a093b01c1e145a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:28:35.017300 23671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4bfe2b06734ef0a537ce962ca942bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:28:35.017278 24047 raft_consensus.cc:348] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 } } peers { permanent_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42369 } } peers { permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 } }
I20260925 02:28:35.017422 24047 raft_consensus.cc:374] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:28:35.017447 24047 raft_consensus.cc:729] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8296db03f8584f6e82eff3454aa90339, State: Initialized, Role: FOLLOWER
I20260925 02:28:35.017541 24047 consensus_queue.cc:260] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 } } peers { permanent_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42369 } } peers { permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 } }
I20260925 02:28:35.017750 24047 ts_tablet_manager.cc:1428] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:28:35.017787 24026 heartbeater.cc:503] Master 127.11.220.254:41879 was elected leader, sending a full tablet report...
I20260925 02:28:35.020273 24051 tablet_bootstrap.cc:492] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a: Bootstrap starting.
I20260925 02:28:35.021315 24051 tablet_bootstrap.cc:654] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a: Neither blocks nor log segments found. Creating new log.
I20260925 02:28:35.021689 24051 log.cc:824] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:28:35.022424 24051 tablet_bootstrap.cc:492] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a: No bootstrap required, opened a new log
I20260925 02:28:35.022482 24051 ts_tablet_manager.cc:1397] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:28:35.024272 24051 raft_consensus.cc:348] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 } } peers { permanent_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42369 } } peers { permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 } }
I20260925 02:28:35.024422 24051 raft_consensus.cc:374] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:28:35.024456 24051 raft_consensus.cc:729] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c292d43594e48f8a1cd30d858f1ca9a, State: Initialized, Role: FOLLOWER
I20260925 02:28:35.024999 24051 consensus_queue.cc:260] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 } } peers { permanent_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42369 } } peers { permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 } }
I20260925 02:28:35.025257 24051 ts_tablet_manager.cc:1428] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 02:28:35.029135 24046 tablet_bootstrap.cc:492] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: No bootstrap required, opened a new log
I20260925 02:28:35.029202 24046 ts_tablet_manager.cc:1397] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.002s
I20260925 02:28:35.031677 23743 heartbeater.cc:503] Master 127.11.220.254:41879 was elected leader, sending a full tablet report...
I20260925 02:28:35.031368 24046 raft_consensus.cc:348] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 } } peers { permanent_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42369 } } peers { permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 } }
I20260925 02:28:35.033249 24046 raft_consensus.cc:374] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:28:35.033289 24046 raft_consensus.cc:729] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb47b958dc8446a8dc5c3bb349bbf53, State: Initialized, Role: FOLLOWER
I20260925 02:28:35.037500 24046 consensus_queue.cc:260] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 } } peers { permanent_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42369 } } peers { permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 } }
I20260925 02:28:35.037804 24046 ts_tablet_manager.cc:1428] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: Time spent starting tablet: real 0.009s user 0.000s sys 0.003s
I20260925 02:28:35.037940 23889 heartbeater.cc:503] Master 127.11.220.254:41879 was elected leader, sending a full tablet report...
Sep 25 02:28:35 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
W20260925 02:28:35.220247 24027 tablet.cc:2369] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:28:35.227896 23890 tablet.cc:2369] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:28:35.272631 23744 tablet.cc:2369] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:28:35.293519 24054 raft_consensus.cc:482] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:28:35.293661 24054 raft_consensus.cc:504] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 } } peers { permanent_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42369 } } peers { permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 } }
I20260925 02:28:35.294013 24054 leader_election.cc:288] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8296db03f8584f6e82eff3454aa90339 (127.11.220.195:35133), 3c292d43594e48f8a1cd30d858f1ca9a (127.11.220.193:39961)
I20260925 02:28:35.297892 24045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.294493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:52911 (local address 127.11.220.195:35133)
0925 02:28:35.294540 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:35.294542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:35.294632 (+ 90us) server_negotiation.cc:407] Connection header received
0925 02:28:35.294733 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:35.294734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:35.294746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:35.294773 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:28:35.294995 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.295680 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.297027 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.297505 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.297567 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.297576 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:35.297781 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 02:28:35.297818 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:28:35.298290 24057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.294195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:35133 (local address 127.11.220.194:52911)
0925 02:28:35.294566 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0925 02:28:35.294577 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:28:35.294660 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:28:35.294846 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:28:35.294847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:28:35.294849 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:28:35.294916 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:35.294918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.295800 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.295800 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:35.296904 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:35.296907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.297634 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.297635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:35.297701 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.297704 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:28:35.298174 (+ 470us) client_negotiation.cc:240] Negotiation successful
0925 02:28:35.298219 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":53,"threads_started":1}
I20260925 02:28:35.298722 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4bfe2b06734ef0a537ce962ca942bd" candidate_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8296db03f8584f6e82eff3454aa90339" is_pre_election: true
I20260925 02:28:35.298875 23960 raft_consensus.cc:2454] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cb47b958dc8446a8dc5c3bb349bbf53 in term 0.
I20260925 02:28:35.299099 23774 leader_election.cc:302] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cb47b958dc8446a8dc5c3bb349bbf53, 8296db03f8584f6e82eff3454aa90339; no voters:
I20260925 02:28:35.299216 24054 raft_consensus.cc:2781] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:28:35.299281 24054 raft_consensus.cc:482] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:28:35.299301 24054 raft_consensus.cc:3037] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:28:35.300271 24054 raft_consensus.cc:504] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 } } peers { permanent_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42369 } } peers { permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 } }
I20260925 02:28:35.300381 24054 leader_election.cc:288] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [CANDIDATE]: Term 1 election: Requested vote from peers 8296db03f8584f6e82eff3454aa90339 (127.11.220.195:35133), 3c292d43594e48f8a1cd30d858f1ca9a (127.11.220.193:39961)
I20260925 02:28:35.300676 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4bfe2b06734ef0a537ce962ca942bd" candidate_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8296db03f8584f6e82eff3454aa90339"
I20260925 02:28:35.300750 23960 raft_consensus.cc:3037] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:28:35.301544 23960 raft_consensus.cc:2454] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cb47b958dc8446a8dc5c3bb349bbf53 in term 1.
I20260925 02:28:35.301759 23774 leader_election.cc:302] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cb47b958dc8446a8dc5c3bb349bbf53, 8296db03f8584f6e82eff3454aa90339; no voters:
I20260925 02:28:35.301853 24054 raft_consensus.cc:2781] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:28:35.301961 24054 raft_consensus.cc:686] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 1 LEADER]: Becoming Leader. State: Replica: 7cb47b958dc8446a8dc5c3bb349bbf53, State: Running, Role: LEADER
I20260925 02:28:35.302037 24054 consensus_queue.cc:237] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 } } peers { permanent_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42369 } } peers { permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 } }
I20260925 02:28:35.302832 23521 catalog_manager.cc:5495] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cb47b958dc8446a8dc5c3bb349bbf53 (127.11.220.194). New cstate: current_term: 1 leader_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 } health_report { overall_health: UNKNOWN } } }
I20260925 02:28:35.304584 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.294361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:39961 (local address 127.11.220.194:48447)
0925 02:28:35.298157 (+ 3796us) negotiation.cc:107] Waiting for socket to connect
0925 02:28:35.298168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:28:35.298252 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:28:35.298412 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:28:35.298412 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:28:35.298414 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:28:35.298464 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:35.298465 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.302609 (+ 4144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.302610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:35.303566 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:28:35.303568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.304331 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.304332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:28:35.304397 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.304400 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:28:35.304474 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 02:28:35.304510 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3780,"thread_start_us":37,"threads_started":1}
I20260925 02:28:35.304610 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.294429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:48447 (local address 127.11.220.193:39961)
0925 02:28:35.294480 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:35.294482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:35.298226 (+ 3744us) server_negotiation.cc:407] Connection header received
0925 02:28:35.298317 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:35.298318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:35.298329 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:35.298356 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:28:35.298538 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.299110 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.303690 (+ 4580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.304176 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.304219 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.304226 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:35.304492 (+ 266us) server_negotiation.cc:299] Negotiation successful
0925 02:28:35.304554 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 02:28:35.305224 23694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4bfe2b06734ef0a537ce962ca942bd" candidate_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" is_pre_election: true
I20260925 02:28:35.305361 23694 raft_consensus.cc:2454] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cb47b958dc8446a8dc5c3bb349bbf53 in term 0.
I20260925 02:28:35.305466 23692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4bfe2b06734ef0a537ce962ca942bd" candidate_uuid: "7cb47b958dc8446a8dc5c3bb349bbf53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c292d43594e48f8a1cd30d858f1ca9a"
I20260925 02:28:35.305516 23692 raft_consensus.cc:3037] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:28:35.306214 23692 raft_consensus.cc:2454] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cb47b958dc8446a8dc5c3bb349bbf53 in term 1.
Sep 25 02:28:35 dist-test-slave-dn4n krb5kdc[23032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303314, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:28:35.369274 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.366747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.11.220.194:42369)
0925 02:28:35.366797 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 02:28:35.366799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:28:35.366808 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:28:35.366851 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:28:35.366852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:28:35.366874 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:28:35.366955 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:28:35.367139 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.367595 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.368350 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:28:35.368581 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:28:35.368665 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:28:35.368820 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:28:35.368962 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:28:35.369062 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 02:28:35.369172 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 02:28:35.369198 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 02:28:35.371389 23692 raft_consensus.cc:1261] T be4bfe2b06734ef0a537ce962ca942bd P 3c292d43594e48f8a1cd30d858f1ca9a [term 1 FOLLOWER]: Refusing update from remote peer 7cb47b958dc8446a8dc5c3bb349bbf53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:28:35.371742 23960 raft_consensus.cc:1261] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339 [term 1 FOLLOWER]: Refusing update from remote peer 7cb47b958dc8446a8dc5c3bb349bbf53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:28:35.372071 24054 consensus_queue.cc:1035] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8296db03f8584f6e82eff3454aa90339" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 35133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:28:35.372277 24059 consensus_queue.cc:1035] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c292d43594e48f8a1cd30d858f1ca9a" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:28:35.375074 24063 mvcc.cc:205] Tried to move back new op lower bound from 7333082379757740032 to 7333082379477409792. Current Snapshot: MvccSnapshot[applied={T|T < 7333082379757740032}]
I20260925 02:28:35.376729 24064 mvcc.cc:205] Tried to move back new op lower bound from 7333082379757740032 to 7333082379477409792. Current Snapshot: MvccSnapshot[applied={T|T < 7333082379757740032}]
I20260925 02:28:35.379380 23521 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:28:35.379487 23521 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:28:35.380501 23521 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b78c07224a346cb8da9bc0d7f7b3eb0: Sending DeleteTablet for 3 replicas of tablet be4bfe2b06734ef0a537ce962ca942bd
I20260925 02:28:35.380822 23802 tablet_service.cc:1514] Processing DeleteTablet for tablet be4bfe2b06734ef0a537ce962ca942bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:28:35 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:56382
I20260925 02:28:35.381304 23939 tablet_service.cc:1514] Processing DeleteTablet for tablet be4bfe2b06734ef0a537ce962ca942bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:28:35 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:55100
I20260925 02:28:35.381609 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 23602
I20260925 02:28:35.384611 24073 tablet_replica.cc:330] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: stopping tablet replica
I20260925 02:28:35.384722 24073 raft_consensus.cc:2229] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:28:35.384809 24073 raft_consensus.cc:2258] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:28:35.385157 24073 ts_tablet_manager.cc:1905] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:28:35.386831 24073 ts_tablet_manager.cc:1918] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:28:35.386915 24073 log.cc:1196] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/be4bfe2b06734ef0a537ce962ca942bd
I20260925 02:28:35.387151 24073 ts_tablet_manager.cc:1939] T be4bfe2b06734ef0a537ce962ca942bd P 7cb47b958dc8446a8dc5c3bb349bbf53: Deleting consensus metadata
I20260925 02:28:35.387493 23505 catalog_manager.cc:4841] TS 7cb47b958dc8446a8dc5c3bb349bbf53 (127.11.220.194:42369): tablet be4bfe2b06734ef0a537ce962ca942bd (table test-table [id=8e2269dc50414f22a093b01c1e145a2d]) successfully deleted
I20260925 02:28:35.394383 24074 tablet_replica.cc:330] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: stopping tablet replica
I20260925 02:28:35.396601 24074 raft_consensus.cc:2229] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:28:35.396777 24074 raft_consensus.cc:2258] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:28:35.397284 24074 ts_tablet_manager.cc:1905] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:28:35.400682 24074 ts_tablet_manager.cc:1918] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:28:35.400803 24074 log.cc:1196] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/be4bfe2b06734ef0a537ce962ca942bd
I20260925 02:28:35.401433 24074 ts_tablet_manager.cc:1939] T be4bfe2b06734ef0a537ce962ca942bd P 8296db03f8584f6e82eff3454aa90339: Deleting consensus metadata
W20260925 02:28:35.403458 23507 connection.cc:535] client connection to 127.11.220.193:39961 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 02:28:35.403564 23507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 02:28:35.404655 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 23752
I20260925 02:28:35.407866 23507 catalog_manager.cc:4841] TS 8296db03f8584f6e82eff3454aa90339 (127.11.220.195:35133): tablet be4bfe2b06734ef0a537ce962ca942bd (table test-table [id=8e2269dc50414f22a093b01c1e145a2d]) successfully deleted
I20260925 02:28:35.408761 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:28:35.408622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:39961 (local address 127.0.0.1:36806)
0925 02:28:35.408670 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 02:28:35.408727 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:39961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260925 02:28:35.408895 23507 catalog_manager.cc:4584] TS 3c292d43594e48f8a1cd30d858f1ca9a (127.11.220.193:39961): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be4bfe2b06734ef0a537ce962ca942bd: Network error: Client connection negotiation failed: client connection to 127.11.220.193:39961: connect: Connection refused (error 111)
I20260925 02:28:35.412719 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 23894
I20260925 02:28:35.417128 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 23489
I20260925 02:28:35.421803 12147 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 02:28:35.767313 12147 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 02:28:35.767400 12147 mini_ranger.cc:66] Stopping Ranger...
I20260925 02:28:36.111968 12147 mini_ranger.cc:68] Stopped Ranger
2026-09-25 02:28:36.112 UTC [23062] LOG: received smart shutdown request
2026-09-25 02:28:36.115 UTC [23062] LOG: background worker "logical replication launcher" (PID 23071) exited with exit code 1
2026-09-25 02:28:36.115 UTC [23066] LOG: shutting down
2026-09-25 02:28:36.272 UTC [23062] LOG: database system is shut down
2026-09-25T02:28:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (42447 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:28:36 dist-test-slave-dn4n krb5kdc[24085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:28:36 dist-test-slave-dn4n krb5kdc[24085](info): set up 2 sockets
Sep 25 02:28:36 dist-test-slave-dn4n krb5kdc[24085](info): commencing operation
krb5kdc: starting...
W20260925 02:28:37.345002 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:28:37 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:28:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:28:37Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.220.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.220.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/rangeradmin_127.11.220.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/rangeradmin_127.11.220.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.220.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.220.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/rangerlookup_127.11.220.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/rangerlookup_127.11.220.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.220.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.220.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/HTTP_127.11.220.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/HTTP_127.11.220.212@KRBTEST.COM.keytab.
I20260925 02:28:37.488859 12147 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfV0OY1/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/postgres -l logfile start
2026-09-25 02:28:39.722 UTC [24117] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 02:28:39.722 UTC [24117] LOG: listening on IPv4 address "127.11.220.212", port 48023
2026-09-25 02:28:39.726 UTC [24117] LOG: listening on Unix socket "/tmp/.s.PGSQL.48023"
2026-09-25 02:28:39.758 UTC [24120] LOG: database system was shut down at 2026-09-25 02:28:38 UTC
2026-09-25 02:28:39.762 UTC [24117] LOG: database system is ready to accept connections
I20260925 02:28:40.665058 12147 mini_postgres.cc:96] Postgres bound to 48023
2026-09-25 02:28:40.670 UTC [24129] FATAL: database "slave" does not exist
127.11.220.212:48023 - accepting connections
I20260925 02:28:40.671312 12147 mini_ranger.cc:161] Starting Ranger...
I20260925 02:28:40.684917 12147 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 02:28:41.118589 12147 mini_ranger.cc:87] Created ranger Postgres database
I20260925 02:28:41.118690 12147 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-admin
2026-09-25 02:28:41,346 [I] DB FLAVOR :POSTGRES
2026-09-25 02:28:41,346 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 02:28:41,346 [I] Checking connection..
2026-09-25 02:28:41,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 02:28:41,565 [I] Checking connection passed.
2026-09-25 02:28:41,566 [I] --------- Verifying version history table ---------
2026-09-25 02:28:41,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:28:41,798 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 02:28:41,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:28:42,029 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 02:28:42,030 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 02:28:42,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 02:28:42.230 UTC [24195] WARNING: there is no transaction in progress
2026-09-25 02:28:42,236 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 02:28:42,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 02:28:42,454 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 02:28:42,454 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 02:28:42,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 02:28:42,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 02:28:42,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-dn4n.c.gcp-upstream.internal','N') ;"
2026-09-25 02:28:43,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 02:28:43,347 [I] Table x_portal_user does not exist in database ranger
2026-09-25 02:28:43,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 02:28:43,576 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 02:28:43,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 02:28:43,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 02:28:44,004 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 02:28:44,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 02:28:44.858 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:44.872 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:44.886 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:44.899 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:44.915 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:44.933 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:44.990 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:44.996 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:45.007 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:45.016 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:45.025 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:45.038 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:45.046 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:45.051 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:45.056 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:45.061 UTC [24306] WARNING: there is no transaction in progress
2026-09-25 02:28:45,583 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 02:28:45,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 02:28:45,783 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 02:28:45,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 02:28:46,003 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 02:28:46,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 02:28:46,207 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 02:28:46,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 02:28:46,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 02:28:46,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-dn4n.c.gcp-upstream.internal';"
2026-09-25 02:28:46,867 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 02:28:46,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 02:28:47,086 [I] DB_PATCHES have already been applied
I20260925 02:28:47.091912 12147 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfV0OY1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 02:28:47.091965 12147 mini_ranger.cc:193] Using host: 127.11.220.212
I20260925 02:28:47.094918 12147 mini_ranger.cc:227] Ranger admin URL: http://127.11.220.212:56745
Sep 25, 2026 2:28:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:28:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:28:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 2:28:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 2:28:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 2:28:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 2:28:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 02:28:48.116880 12147 mini_ranger.cc:160] Time spent starting Ranger: real 7.446s user 0.000s sys 0.008s
Sep 25, 2026 2:29:06 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 2:29:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Sep 25, 2026 2:29:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 2:29:06 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 2:29:07 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 02:29:07.806308 12147 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.220.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.220.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/rangerkms_127.11.220.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.220.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/rangerkms_127.11.220.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.220.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.220.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.220.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/HTTP_127.11.220.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.220.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/HTTP_127.11.220.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 02:29:07 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303347, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 02:29:07.881700 12147 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 02:29:07.889614 12147 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 02:29:08.595865 12147 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 02:29:08.595956 12147 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-kms
I20260925 02:29:08.595974 12147 mini_ranger_kms.cc:227] Using postgres at 127.11.220.212:48023
2026-09-25 02:29:08,748 [I] DB FLAVOR :POSTGRES
2026-09-25 02:29:08,749 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 02:29:08,749 [I] Checking connection
2026-09-25 02:29:08,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 02:29:08,950 [I] connection success
2026-09-25 02:29:08,950 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 02:29:08,950 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 02:29:08,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 02:29:09,156 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 02:29:09,156 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 02:29:09,156 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 02:29:09,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.220.212:48023/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 02:29:09,408 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 02:29:09.834054 12147 mini_ranger_kms.cc:310] Created kms service
I20260925 02:29:10.000677 12147 mini_ranger_kms.cc:326] Created kudu user
I20260925 02:29:10.039287 12147 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 02:29:10 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303347, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:29:10.390403 12147 mini_ranger_kms.cc:384] Added ranger policy
I20260925 02:29:10.390507 12147 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfV0OY1/build/release/bin/postgresql.jar:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfV0OY1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 02:29:10.390525 12147 mini_ranger_kms.cc:240] Using host: 127.11.220.212
I20260925 02:29:10.393677 12147 mini_ranger_kms.cc:276] Ranger KMS PID: 24488
I20260925 02:29:10.393755 12147 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.220.212:37577
Sep 25, 2026 2:29:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:29:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:29:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 2:29:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 2:29:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 2:29:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 25 02:29:12 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790303352, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.220.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:12.500644 12147 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.619s user 0.000s sys 0.137s
I20260925 02:29:12.500736 12147 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 02:29:12.500778 12147 mini_ranger_kms.cc:401] 127.11.220.212:37577/kms/v1/keys
Sep 25 02:29:12 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303352, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.220.212@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
Sep 25 02:29:13 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303352, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.220.212@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
Sep 25 02:29:15 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:29:15.037431 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:36483
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:36483
--ranger_config_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:37577/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:15.110203 24541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:15.110409 24541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:15.110440 24541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:15.111490 24541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:29:15.111550 24541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:15.111577 24541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:15.111601 24541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:29:15.111630 24541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:29:15.113190 24541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.220.212:37577/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:36483
--ranger_config_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:36483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:15.113454 24541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:15.113667 24541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:15.118412 24546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:15.118427 24547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:15.118427 24550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:15.118563 24541 server_base.cc:1034] running on GCE node
I20260925 02:29:15.118788 24541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:15.119042 24541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:15.120198 24541 hybrid_clock.cc:648] HybridClock initialized: now 1790303355120186 us; error 30 us; skew 500 ppm
Sep 25 02:29:15 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303355, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:15.122756 24541 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:29:15.123168 24541 webserver.cc:460] Webserver started at http://127.11.220.254:37263/ using document root <none> and password file <none>
I20260925 02:29:15.123381 24541 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:15.123440 24541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:15.123596 24541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:29:15 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303355, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:29:15.163542 24541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "47030303b3b44ed3934bdd8318a66373"
format_stamp: "Formatted at 2026-09-25 02:29:15 on dist-test-slave-dn4n"
server_key: "d9f2690f4bcd9be35576044572dcb39a"
server_key_iv: "7d5c914e595a04652039bee716912ec9"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:15.163980 24541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "47030303b3b44ed3934bdd8318a66373"
format_stamp: "Formatted at 2026-09-25 02:29:15 on dist-test-slave-dn4n"
server_key: "d9f2690f4bcd9be35576044572dcb39a"
server_key_iv: "7d5c914e595a04652039bee716912ec9"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:15.165413 24541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:29:15.175521 24557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:15.175707 24541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:15.175791 24541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "47030303b3b44ed3934bdd8318a66373"
format_stamp: "Formatted at 2026-09-25 02:29:15 on dist-test-slave-dn4n"
server_key: "d9f2690f4bcd9be35576044572dcb39a"
server_key_iv: "7d5c914e595a04652039bee716912ec9"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:15.175880 24541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:15.186517 24541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:15.188109 24541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:15.188246 24541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:15.192718 24541 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:36483
I20260925 02:29:15.192750 24610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:36483 every 8 connection(s)
I20260925 02:29:15.193114 24541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:29:15.193740 24611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:15.195780 24611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: Bootstrap starting.
I20260925 02:29:15.196310 24611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:15.196633 24611 log.cc:824] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:15.197286 24611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: No bootstrap required, opened a new log
I20260925 02:29:15.199028 24611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } }
I20260925 02:29:15.199148 24611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:15.199168 24611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47030303b3b44ed3934bdd8318a66373, State: Initialized, Role: FOLLOWER
I20260925 02:29:15.199297 24611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } }
I20260925 02:29:15.199358 24611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:29:15.199376 24611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:29:15.199434 24611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:15.199905 24611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } }
I20260925 02:29:15.200016 24611 leader_election.cc:302] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47030303b3b44ed3934bdd8318a66373; no voters:
I20260925 02:29:15.200189 24611 leader_election.cc:288] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:29:15.200223 24614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:29:15.200368 24614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 1 LEADER]: Becoming Leader. State: Replica: 47030303b3b44ed3934bdd8318a66373, State: Running, Role: LEADER
I20260925 02:29:15.200425 24611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:29:15.200507 24614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } }
I20260925 02:29:15.200908 24616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47030303b3b44ed3934bdd8318a66373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } } }
I20260925 02:29:15.200913 24614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47030303b3b44ed3934bdd8318a66373. Latest consensus state: current_term: 1 leader_uuid: "47030303b3b44ed3934bdd8318a66373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } } }
I20260925 02:29:15.201050 24616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:15.201076 24614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:15.201737 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 24541
I20260925 02:29:15.201828 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:29:15.201920 24611 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 02:29:15.201980 12147 external_mini_cluster.cc:1428] Setting key f3d8432561e7b1c97f5c2e6f58f699b0
I20260925 02:29:15.202406 24624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:29:15.202930 24624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:29:15.207109 24624 catalog_manager.cc:1347] Generated new cluster ID: 3ffdf76f37ce40f9bb5fd46d28040771
I20260925 02:29:15.207196 24624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:29:15 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:15.225209 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:15.202379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.11.220.254:36483)
0925 02:29:15.215628 (+ 13249us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:15.215637 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:15.215668 (+ 31us) server_negotiation.cc:407] Connection header received
0925 02:29:15.215893 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:15.215900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:15.215974 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:15.216131 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:15.216865 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:15.218206 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:15.219097 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:15.219648 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:15.219758 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:15.222120 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:15.222124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:15.222124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:15.222135 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:15.223780 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:15.224107 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:15.224110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:15.224110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:15.224208 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:15.224370 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:15.224370 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:15.224371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:15.224651 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:15.224769 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:15.224903 (+ 134us) server_negotiation.cc:299] Negotiation successful
0925 02:29:15.225028 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13197,"thread_start_us":59,"threads_started":1}
I20260925 02:29:15.233671 24624 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:29:15.234351 24624 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:29:15.237834 24624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: Generated new TSK 0
I20260925 02:29:15.238010 24624 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:29:15.943092 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:37577/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36483
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:16.024785 24658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:16.024977 24658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:16.025007 24658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:16.025990 24658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:16.026049 24658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:16.026086 24658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:29:16.027699 24658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.220.212:37577/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:16.027916 24658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:16.028409 24658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:16.034358 24663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:16.034487 24664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:16.034641 24666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:16.035910 24658 server_base.cc:1034] running on GCE node
I20260925 02:29:16.036126 24658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:16.036371 24658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:16.037520 24658 hybrid_clock.cc:648] HybridClock initialized: now 1790303356037490 us; error 41 us; skew 500 ppm
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:16.041123 24658 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:29:16.041616 24658 webserver.cc:460] Webserver started at http://127.11.220.193:37211/ using document root <none> and password file <none>
I20260925 02:29:16.041888 24658 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:16.041983 24658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:16.042308 24658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:29:16.061153 24658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "5278172fee7e4850851481a8125c8923"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "fec09bc2c31f6b5bacfb23863f4af42e"
server_key_iv: "d610f7aa69d329472232743d3fe20202"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.061554 24658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "5278172fee7e4850851481a8125c8923"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "fec09bc2c31f6b5bacfb23863f4af42e"
server_key_iv: "d610f7aa69d329472232743d3fe20202"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.068775 24658 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260925 02:29:16.079684 24673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.079880 24658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:16.079963 24658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "5278172fee7e4850851481a8125c8923"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "fec09bc2c31f6b5bacfb23863f4af42e"
server_key_iv: "d610f7aa69d329472232743d3fe20202"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.080052 24658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:16.102969 24658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:16.105424 24658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:16.105626 24658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:16.105871 24658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:16.106276 24658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:16.106379 24658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.106472 24658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:16.106554 24658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.113287 24658 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:33471
I20260925 02:29:16.113759 24658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:29:16.118862 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 24658
I20260925 02:29:16.118960 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:29:16.119138 12147 external_mini_cluster.cc:1428] Setting key d4eab1e8e935417186d109ac1560de04
I20260925 02:29:16.114319 24793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:33471 every 8 connection(s)
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:16.124712 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:16.114742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:35821 (local address 127.11.220.254:36483)
0925 02:29:16.115698 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:16.115699 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:16.115707 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:29:16.115732 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:16.115733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:16.115745 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:16.115805 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:16.116548 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.117246 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.118641 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.118919 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.118984 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:16.121344 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:16.121345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:16.121345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:16.121351 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:16.122994 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:16.123411 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:16.123412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:16.123412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:16.123469 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:16.123730 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:16.123731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:16.123731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:16.123862 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:16.123914 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:16.124546 (+ 632us) server_negotiation.cc:299] Negotiation successful
0925 02:29:16.124623 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":923,"thread_start_us":66,"threads_started":1}
I20260925 02:29:16.124872 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:16.114455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:36483 (local address 127.11.220.193:35821)
0925 02:29:16.114883 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:16.114926 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:16.115507 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:16.115930 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:16.115935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:16.116149 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:16.116395 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:16.116401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.117349 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.117350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:16.118545 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:16.118549 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.119063 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.119064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:16.119219 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:16.119766 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:16.119775 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:16.121252 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:16.123078 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:16.123084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:16.123086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:16.123333 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:16.123538 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:16.123538 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:16.123539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:16.123645 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:16.123938 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:16.123943 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:16.124078 (+ 135us) client_negotiation.cc:769] Sending connection context
0925 02:29:16.124173 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 02:29:16.124298 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":81,"threads_started":1}
I20260925 02:29:16.125402 24794 heartbeater.cc:347] Connected to a master server at 127.11.220.254:36483
I20260925 02:29:16.125479 24794 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:16.125643 24794 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
I20260925 02:29:16.126124 24575 ts_manager.cc:194] Registered new tserver with Master: 5278172fee7e4850851481a8125c8923 (127.11.220.193:33471)
I20260925 02:29:16.126909 24575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:35821
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:29:16.145185 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:37577/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36483
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:16.255761 24800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:16.256100 24800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:16.256196 24800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:16.257822 24800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:16.257936 24800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:16.258034 24800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:29:16.260182 24800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.220.212:37577/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:16.260710 24800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:16.261042 24800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:16.267238 24808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:16.267575 24807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:16.267820 24810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:16.268926 24800 server_base.cc:1034] running on GCE node
I20260925 02:29:16.269150 24800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:16.269389 24800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:16.270573 24800 hybrid_clock.cc:648] HybridClock initialized: now 1790303356270553 us; error 36 us; skew 500 ppm
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:16.273917 24800 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:29:16.274457 24800 webserver.cc:460] Webserver started at http://127.11.220.194:44683/ using document root <none> and password file <none>
I20260925 02:29:16.274731 24800 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:16.274796 24800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:16.274914 24800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:16.298969 24800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "f95da849282f4926a4738e2d9b334694"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "659e0f2b68366c9f6ff3e9cd3dd9567b"
server_key_iv: "1dcc52a04930fc21a55c89e81a2f2177"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.299508 24800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "f95da849282f4926a4738e2d9b334694"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "659e0f2b68366c9f6ff3e9cd3dd9567b"
server_key_iv: "1dcc52a04930fc21a55c89e81a2f2177"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.304786 24800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.002s
I20260925 02:29:16.322717 24817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.322976 24800 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260925 02:29:16.323101 24800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "f95da849282f4926a4738e2d9b334694"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "659e0f2b68366c9f6ff3e9cd3dd9567b"
server_key_iv: "1dcc52a04930fc21a55c89e81a2f2177"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.323189 24800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:16.339608 24800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:16.341806 24800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:16.342087 24800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:16.342350 24800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:16.342744 24800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:16.342839 24800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.342917 24800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:16.342980 24800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.350361 24800 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:38923
I20260925 02:29:16.350814 24800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:29:16.352873 24933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:38923 every 8 connection(s)
I20260925 02:29:16.359261 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 24800
I20260925 02:29:16.359367 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:29:16.359567 12147 external_mini_cluster.cc:1428] Setting key 4fb42501421c46b545d9c3e717f37c51
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:16.366158 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:16.353804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:45089 (local address 127.11.220.254:36483)
0925 02:29:16.353856 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:16.353858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:16.354121 (+ 263us) server_negotiation.cc:407] Connection header received
0925 02:29:16.354887 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:16.354889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:16.354906 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:16.354938 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:16.355770 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.356536 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.358033 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.358352 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.358396 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:16.363130 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:16.363131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:16.363132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:16.363138 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:16.364791 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:16.365234 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:16.365236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:16.365236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:16.365284 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:16.365500 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:16.365501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:16.365501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:16.365613 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:16.365649 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:16.365976 (+ 327us) server_negotiation.cc:299] Negotiation successful
0925 02:29:16.366066 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 02:29:16.366570 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:16.353515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:36483 (local address 127.11.220.194:45089)
0925 02:29:16.353980 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:16.354035 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:16.354727 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:16.355158 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:16.355162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:16.355396 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:16.355660 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:16.355665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.356648 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.356649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:16.357918 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:16.357920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.358506 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.358507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:16.358645 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:16.361565 (+ 2920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:16.361577 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:16.363030 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:16.364891 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:16.364897 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:16.364899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:16.365160 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:16.365343 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:16.365343 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:16.365344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:16.365441 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:16.365687 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:16.365691 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:16.365836 (+ 145us) client_negotiation.cc:769] Sending connection context
0925 02:29:16.366235 (+ 399us) client_negotiation.cc:240] Negotiation successful
0925 02:29:16.366368 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":84,"threads_started":1}
I20260925 02:29:16.366979 24934 heartbeater.cc:347] Connected to a master server at 127.11.220.254:36483
I20260925 02:29:16.367091 24934 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:16.367275 24934 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
I20260925 02:29:16.367615 24575 ts_manager.cc:194] Registered new tserver with Master: f95da849282f4926a4738e2d9b334694 (127.11.220.194:38923)
I20260925 02:29:16.368098 24575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:45089
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:29:16.395277 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:37577/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36483
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:16.505560 24940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:16.505792 24940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:16.505825 24940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:16.507225 24940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:16.507278 24940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:16.507310 24940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:29:16.509126 24940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.220.212:37577/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:36483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:16.509326 24940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:16.509549 24940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:16.515172 24945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:16.515180 24946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:16.515445 24948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:16.516093 24940 server_base.cc:1034] running on GCE node
I20260925 02:29:16.516263 24940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:16.516500 24940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:16.517668 24940 hybrid_clock.cc:648] HybridClock initialized: now 1790303356517643 us; error 37 us; skew 500 ppm
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:16.520587 24940 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:29:16.521040 24940 webserver.cc:460] Webserver started at http://127.11.220.195:38185/ using document root <none> and password file <none>
I20260925 02:29:16.521278 24940 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:16.521341 24940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:16.521515 24940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:29:16.536177 24940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "72da224d63df4dd581704ca71adb1f89"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "bbe17b5db6b5c5e07d34bbccd447a7f7"
server_key_iv: "8019f679159dd830eb40803fbf4a4fa3"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.536674 24940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "72da224d63df4dd581704ca71adb1f89"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "bbe17b5db6b5c5e07d34bbccd447a7f7"
server_key_iv: "8019f679159dd830eb40803fbf4a4fa3"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.539393 24940 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260925 02:29:16.555898 24955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.556049 24940 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260925 02:29:16.556125 24940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "72da224d63df4dd581704ca71adb1f89"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "bbe17b5db6b5c5e07d34bbccd447a7f7"
server_key_iv: "8019f679159dd830eb40803fbf4a4fa3"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.556198 24940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:16.574836 24940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:16.576715 24940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:16.576859 24940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:16.577055 24940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:16.577368 24940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:16.577421 24940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.577454 24940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:16.577477 24940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.584160 24940 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:44347
I20260925 02:29:16.584594 24940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:29:16.584683 25071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:44347 every 8 connection(s)
I20260925 02:29:16.591536 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 24940
I20260925 02:29:16.591645 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:29:16.591842 12147 external_mini_cluster.cc:1428] Setting key 91cb51779c9fefca571e91e6fe6d8ddd
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:16.609748 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:16.592199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:46129 (local address 127.11.220.254:36483)
0925 02:29:16.592249 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:16.592250 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:16.595497 (+ 3247us) server_negotiation.cc:407] Connection header received
0925 02:29:16.596172 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:16.596174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:16.596187 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:16.596214 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:16.597052 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.597740 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.599123 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.599397 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.599436 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:16.606885 (+ 7449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:16.606886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:16.606887 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:16.606894 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:16.608326 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:16.608776 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:16.608778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:16.608778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:16.608830 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:16.609131 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:16.609132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:16.609132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:16.609246 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:16.609291 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:16.609599 (+ 308us) server_negotiation.cc:299] Negotiation successful
0925 02:29:16.609668 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 02:29:16.610409 25074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:16.592233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:36483 (local address 127.11.220.195:46129)
0925 02:29:16.595375 (+ 3142us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:16.595420 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:16.596062 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:16.596384 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:16.596388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:16.596653 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:16.596927 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:16.596933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.597849 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.597850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:16.599030 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:16.599033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.599531 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.599532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:16.599652 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:16.605354 (+ 5702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:16.605363 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:16.606689 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:16.608414 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:16.608421 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:16.608422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:16.608696 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:16.608901 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:16.608902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:16.608902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:16.609059 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:16.609320 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:16.609325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:16.609456 (+ 131us) client_negotiation.cc:769] Sending connection context
0925 02:29:16.610062 (+ 606us) client_negotiation.cc:240] Negotiation successful
0925 02:29:16.610196 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3099,"thread_start_us":84,"threads_started":1}
I20260925 02:29:16.610976 25072 heartbeater.cc:347] Connected to a master server at 127.11.220.254:36483
I20260925 02:29:16.611101 25072 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:16.611295 25072 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
I20260925 02:29:16.611609 24575 ts_manager.cc:194] Registered new tserver with Master: 72da224d63df4dd581704ca71adb1f89 (127.11.220.195:44347)
I20260925 02:29:16.612010 24575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:46129
I20260925 02:29:16.615444 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:29:16.616197 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 24658
I20260925 02:29:16.621822 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 24800
I20260925 02:29:16.628180 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 24940
I20260925 02:29:16.634732 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 24541
I20260925 02:29:16.640420 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:36483
--webserver_interface=127.11.220.254
--webserver_port=37263
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:36483
--ranger_config_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:37577/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.220.212:37577/kms with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:16.714704 25075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:16.714913 25075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:16.714950 25075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:16.716030 25075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:29:16.716089 25075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:16.716109 25075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:16.716128 25075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:29:16.716154 25075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:29:16.717612 25075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.220.212:37577/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:36483
--ranger_config_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:36483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=37263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:16.717847 25075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:16.718065 25075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:16.723338 25080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:16.723330 25081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:16.723330 25083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:16.723574 25075 server_base.cc:1034] running on GCE node
I20260925 02:29:16.723805 25075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:16.724097 25075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:16.725246 25075 hybrid_clock.cc:648] HybridClock initialized: now 1790303356725234 us; error 31 us; skew 500 ppm
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:16.727454 25075 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:29:16.727878 25075 webserver.cc:460] Webserver started at http://127.11.220.254:37263/ using document root <none> and password file <none>
I20260925 02:29:16.728082 25075 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:16.728150 25075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:29:16.785892 25075 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:16.786553 25091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.786733 25075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:16.786803 25075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "47030303b3b44ed3934bdd8318a66373"
format_stamp: "Formatted at 2026-09-25 02:29:15 on dist-test-slave-dn4n"
server_key: "d9f2690f4bcd9be35576044572dcb39a"
server_key_iv: "7d5c914e595a04652039bee716912ec9"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.787129 25075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:16.798312 25075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:16.799638 25075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:16.799775 25075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:16.803648 25075 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:36483
I20260925 02:29:16.803694 25144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:36483 every 8 connection(s)
I20260925 02:29:16.804008 25075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:29:16.805090 25146 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:29:16.805665 25146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: Bootstrap starting.
I20260925 02:29:16.805969 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 25075
I20260925 02:29:16.806202 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:37577/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:33471
--local_ip_for_outbound_sockets=127.11.220.193
--tserver_master_addrs=127.11.220.254:36483
--webserver_port=37211
--webserver_interface=127.11.220.193
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.220.212:37577/kms with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:29:16.813575 25146 log.cc:824] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:16.815069 25146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:29:16.815308 25146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: Bootstrap complete.
I20260925 02:29:16.817631 25146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } }
I20260925 02:29:16.817862 25146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47030303b3b44ed3934bdd8318a66373, State: Initialized, Role: FOLLOWER
I20260925 02:29:16.818051 25146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } }
I20260925 02:29:16.818151 25146 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:29:16.818199 25146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:29:16.818244 25146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:29:16.819115 25146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } }
I20260925 02:29:16.819229 25146 leader_election.cc:302] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47030303b3b44ed3934bdd8318a66373; no voters:
I20260925 02:29:16.819397 25146 leader_election.cc:288] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 02:29:16.819427 25150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 02:29:16.819521 25150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [term 2 LEADER]: Becoming Leader. State: Replica: 47030303b3b44ed3934bdd8318a66373, State: Running, Role: LEADER
I20260925 02:29:16.819631 25146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:29:16.819603 25150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } }
I20260925 02:29:16.819850 25150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47030303b3b44ed3934bdd8318a66373. Latest consensus state: current_term: 2 leader_uuid: "47030303b3b44ed3934bdd8318a66373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } } }
I20260925 02:29:16.819907 25150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:16.820040 25151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "47030303b3b44ed3934bdd8318a66373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47030303b3b44ed3934bdd8318a66373" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 36483 } } }
I20260925 02:29:16.820117 25151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:16.820571 25154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:29:16.821079 25154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:29:16.821259 25154 catalog_manager.cc:1259] Loaded cluster ID: 3ffdf76f37ce40f9bb5fd46d28040771
I20260925 02:29:16.821336 25154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:29:16.821388 25146 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 02:29:16.824764 25154 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:29:16.825060 25154 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: Loaded TSK: 0
I20260925 02:29:16.825277 25154 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:29:16.905597 25148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:16.905830 25148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:16.905867 25148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:16.906955 25148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:16.907012 25148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:16.907073 25148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:29:16.908437 25148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.220.212:37577/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:33471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=37211
--tserver_master_addrs=127.11.220.254:36483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:16.908737 25148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:16.908949 25148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:16.915287 25183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:16.916321 25186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:16.916519 25148 server_base.cc:1034] running on GCE node
W20260925 02:29:16.916615 25184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:16.916873 25148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:16.917150 25148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:16.918300 25148 hybrid_clock.cc:648] HybridClock initialized: now 1790303356918269 us; error 48 us; skew 500 ppm
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:16.921569 25148 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:29:16.922037 25148 webserver.cc:460] Webserver started at http://127.11.220.193:37211/ using document root <none> and password file <none>
I20260925 02:29:16.922250 25148 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:16.922326 25148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:29:16.939682 25148 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260925 02:29:16.942250 25194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.942667 25148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 02:29:16.942750 25148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "5278172fee7e4850851481a8125c8923"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "fec09bc2c31f6b5bacfb23863f4af42e"
server_key_iv: "d610f7aa69d329472232743d3fe20202"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:16.943145 25148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:16.973399 25148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:16.975320 25148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:16.975420 25148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:16.975615 25148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:16.975937 25148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:16.975970 25148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.975991 25148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:16.976002 25148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:16.982852 25148 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:33471
I20260925 02:29:16.982933 25309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:33471 every 8 connection(s)
I20260925 02:29:16.983315 25148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:29:16.985222 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 25148
I20260925 02:29:16.985502 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:37577/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:38923
--local_ip_for_outbound_sockets=127.11.220.194
--tserver_master_addrs=127.11.220.254:36483
--webserver_port=44683
--webserver_interface=127.11.220.194
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.220.212:37577/kms with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:29:16 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303356, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:16.995895 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:16.984180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:55103 (local address 127.11.220.254:36483)
0925 02:29:16.984587 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:16.984595 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:16.984632 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:29:16.985198 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:16.985206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:16.985265 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:16.985396 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:16.987179 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.988288 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.989348 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.989839 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.989959 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:16.992225 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:16.992229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:16.992230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:16.992242 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:16.994108 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:16.994562 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:16.994565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:16.994566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:16.994660 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:16.994904 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:16.994904 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:16.994905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:16.995195 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:16.995312 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:16.995611 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 02:29:16.995713 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":72,"threads_started":1}
I20260925 02:29:16.995922 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:16.984102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:36483 (local address 127.11.220.193:55103)
0925 02:29:16.984434 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:16.984488 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:16.984960 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:16.986721 (+ 1761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:16.986724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:16.986879 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:16.987064 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:16.987068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.988375 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.988377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:16.989251 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:16.989253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:16.989999 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:16.990000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:16.990124 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:16.990610 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:16.990618 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:16.992114 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:16.994200 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:16.994206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:16.994208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:16.994464 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:16.994728 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:16.994729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:16.994729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:16.994838 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:16.995303 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:16.995309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:16.995417 (+ 108us) client_negotiation.cc:769] Sending connection context
0925 02:29:16.995547 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 02:29:16.995699 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":253,"threads_started":1}
I20260925 02:29:16.997063 25310 heartbeater.cc:347] Connected to a master server at 127.11.220.254:36483
I20260925 02:29:16.997156 25310 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:16.997331 25310 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
W20260925 02:29:16.998184 25310 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:36483 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 02:29:16.998260 25310 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:16.998319 25310 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
I20260925 02:29:16.998581 25310 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:16.998644 25310 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
W20260925 02:29:16.998883 25310 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:29:17.093165 25314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:17.093324 25314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:17.093343 25314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:17.094870 25314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:17.094937 25314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:17.094972 25314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:29:17.096558 25314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.220.212:37577/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:38923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=44683
--tserver_master_addrs=127.11.220.254:36483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:17.096786 25314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:17.097020 25314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:17.102414 25319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:17.102392 25320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:17.102392 25322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:17.102952 25314 server_base.cc:1034] running on GCE node
I20260925 02:29:17.103147 25314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:17.103367 25314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:17.104516 25314 hybrid_clock.cc:648] HybridClock initialized: now 1790303357104501 us; error 35 us; skew 500 ppm
Sep 25 02:29:17 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:17.107191 25314 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:29:17.107618 25314 webserver.cc:460] Webserver started at http://127.11.220.194:44683/ using document root <none> and password file <none>
I20260925 02:29:17.107834 25314 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:17.107905 25314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 02:29:17 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:29:17.122448 25314 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:29:17.123255 25331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:17.123435 25314 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 02:29:17.123515 25314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "f95da849282f4926a4738e2d9b334694"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "659e0f2b68366c9f6ff3e9cd3dd9567b"
server_key_iv: "1dcc52a04930fc21a55c89e81a2f2177"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:17.123859 25314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:17.142529 25314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:17.144510 25314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:17.144661 25314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:17.144889 25314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:17.145207 25314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:17.145254 25314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:17.145282 25314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:17.145299 25314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:17.151952 25314 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:38923
I20260925 02:29:17.152278 25314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:29:17.152832 25448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:38923 every 8 connection(s)
I20260925 02:29:17.157099 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 25314
I20260925 02:29:17.157405 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.220.212:37577/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:44347
--local_ip_for_outbound_sockets=127.11.220.195
--tserver_master_addrs=127.11.220.254:36483
--webserver_port=38185
--webserver_interface=127.11.220.195
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.220.212:37577/kms with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:29:17 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:17.161437 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:17.152949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:36483 (local address 127.11.220.194:47659)
0925 02:29:17.153102 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:17.153138 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:17.153704 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:17.153967 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:17.153971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:17.154133 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:17.154350 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:17.154354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:17.155019 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:17.155020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:17.155849 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:17.155852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:17.156265 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:17.156267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:17.156378 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:17.156805 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:17.156822 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:17.158286 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:17.159891 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:17.159898 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:17.159900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:17.160193 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:17.160397 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:17.160397 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:17.160398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:17.160524 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:17.160820 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:17.160825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:17.160963 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 02:29:17.161051 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 02:29:17.161194 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260925 02:29:17.161897 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:17.153236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:47659 (local address 127.11.220.254:36483)
0925 02:29:17.153301 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:17.153303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:17.153314 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:29:17.153825 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:17.153827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:17.153844 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:17.153892 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:17.154438 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:17.154931 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:17.155934 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:17.156157 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:17.156236 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:17.158384 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:17.158386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:17.158387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:17.158394 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:17.159802 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:17.160275 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:17.160276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:17.160276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:17.160336 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:17.160601 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:17.160602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:17.160602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:17.160739 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:17.160796 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:17.161674 (+ 878us) server_negotiation.cc:299] Negotiation successful
0925 02:29:17.161755 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 02:29:17.162297 25449 heartbeater.cc:347] Connected to a master server at 127.11.220.254:36483
I20260925 02:29:17.162385 25449 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:17.162562 25449 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
W20260925 02:29:17.163283 25449 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:36483 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 02:29:17.163357 25449 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:17.163432 25449 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
I20260925 02:29:17.163803 25449 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:17.163897 25449 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
W20260925 02:29:17.164258 25449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:29:17.249838 25452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:17.250067 25452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:17.250102 25452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:17.251104 25452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:17.251161 25452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:17.251201 25452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:29:17.252774 25452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.220.212:37577/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:44347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=38185
--tserver_master_addrs=127.11.220.254:36483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:17.252974 25452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:17.253206 25452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:17.258617 25460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:17.258869 25458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:17.259172 25457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:17.259545 25452 server_base.cc:1034] running on GCE node
I20260925 02:29:17.259722 25452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:17.259953 25452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:17.261123 25452 hybrid_clock.cc:648] HybridClock initialized: now 1790303357261112 us; error 24 us; skew 500 ppm
Sep 25 02:29:17 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:17.263504 25452 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:29:17.263967 25452 webserver.cc:460] Webserver started at http://127.11.220.195:38185/ using document root <none> and password file <none>
I20260925 02:29:17.264215 25452 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:17.264290 25452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 02:29:17 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:29:17.282608 25452 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.000s sys 0.001s
I20260925 02:29:17.283499 25469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:17.283661 25452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:29:17.283746 25452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "72da224d63df4dd581704ca71adb1f89"
format_stamp: "Formatted at 2026-09-25 02:29:16 on dist-test-slave-dn4n"
server_key: "bbe17b5db6b5c5e07d34bbccd447a7f7"
server_key_iv: "8019f679159dd830eb40803fbf4a4fa3"
server_key_version: "kuduclusterkey@0"
I20260925 02:29:17.284104 25452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:17.305052 25452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:17.306766 25452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:17.306907 25452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:17.307133 25452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:17.307449 25452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:17.307507 25452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:17.307547 25452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:17.307572 25452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:17.314404 25452 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:44347
I20260925 02:29:17.314765 25452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:29:17.315274 25586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:44347 every 8 connection(s)
I20260925 02:29:17.315543 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 25452
Sep 25 02:29:17 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:17.325232 25591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:17.316026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.11.220.254:36483)
0925 02:29:17.316846 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:17.316848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:17.316854 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:29:17.316900 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:17.316901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:17.316914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:17.316976 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:17.317749 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:17.318223 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:17.319243 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:17.319526 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:17.319578 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:17.321358 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:17.321360 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:17.321360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:17.321366 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:17.323260 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:17.323884 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:17.323885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:17.323885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:17.323956 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:17.324168 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:17.324169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:17.324169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:17.324292 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:17.324604 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:17.325043 (+ 439us) server_negotiation.cc:299] Negotiation successful
0925 02:29:17.325119 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":52,"threads_started":1}
I20260925 02:29:17.325654 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:17.315756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:56569 (local address 127.11.220.254:36483)
0925 02:29:17.315814 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:17.315816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:17.316288 (+ 472us) server_negotiation.cc:407] Connection header received
0925 02:29:17.316801 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:17.316802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:17.316814 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:17.316891 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:17.317653 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:17.318364 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:17.319589 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:17.319836 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:17.319870 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:17.322453 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:17.322454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:17.322454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:17.322460 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:17.324150 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:17.324641 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:17.324643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:17.324643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:17.324695 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:17.324957 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:17.324958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:17.324959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:17.325075 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:17.325124 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:17.325493 (+ 369us) server_negotiation.cc:299] Negotiation successful
0925 02:29:17.325574 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 02:29:17.326094 25589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:17.315504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:36483 (local address 127.11.220.195:56569)
0925 02:29:17.316145 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:17.316210 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:17.316705 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:17.316989 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:17.316994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:17.317226 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:17.317531 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:17.317536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:17.318444 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:17.318445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:17.319504 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:17.319507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:17.320023 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:17.320025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:17.320177 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:17.320638 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:17.320648 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:17.322357 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:17.324235 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:17.324240 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:17.324242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:17.324557 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:17.324763 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:17.324764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:17.324764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:17.324875 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:17.325156 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:17.325161 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:17.325334 (+ 173us) client_negotiation.cc:769] Sending connection context
0925 02:29:17.325739 (+ 405us) client_negotiation.cc:240] Negotiation successful
0925 02:29:17.325883 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":73,"threads_started":1}
I20260925 02:29:17.326502 25587 heartbeater.cc:347] Connected to a master server at 127.11.220.254:36483
I20260925 02:29:17.326593 25587 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:17.326761 25587 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
W20260925 02:29:17.327432 25587 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:36483 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 02:29:17.327509 25587 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:17.327571 25587 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
I20260925 02:29:17.327895 25587 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:17.327973 25587 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
W20260925 02:29:17.328274 25587 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 02:29:17 dist-test-slave-dn4n krb5kdc[24085](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790303357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:17.999059 25310 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:17.999217 25310 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
I20260925 02:29:17.999552 25109 ts_manager.cc:194] Registered new tserver with Master: 5278172fee7e4850851481a8125c8923 (127.11.220.193:33471)
I20260925 02:29:18.000233 25109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:55103
Sep 25 02:29:18 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:29:18.167411 25449 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:18.167541 25449 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
I20260925 02:29:18.167801 25109 ts_manager.cc:194] Registered new tserver with Master: f95da849282f4926a4738e2d9b334694 (127.11.220.194:38923)
I20260925 02:29:18.168296 25109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:47659
I20260925 02:29:18.328473 25587 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:18.328609 25587 heartbeater.cc:511] Master 127.11.220.254:36483 requested a full tablet report, sending...
I20260925 02:29:18.328850 25109 ts_manager.cc:194] Registered new tserver with Master: 72da224d63df4dd581704ca71adb1f89 (127.11.220.195:44347)
I20260925 02:29:18.329200 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:29:18.329320 25109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:56569
I20260925 02:29:18.337916 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.330352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55500 (local address 127.11.220.254:36483)
0925 02:29:18.330464 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:18.330466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:18.330582 (+ 116us) server_negotiation.cc:407] Connection header received
0925 02:29:18.330640 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:18.330641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:18.330653 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:18.330715 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:18.331313 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.332033 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.333024 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.333289 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.333352 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.334826 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:18.334827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:18.334828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:18.334835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:18.336602 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:18.336947 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:18.336949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:18.336949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:18.337002 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:18.337175 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:18.337176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:18.337177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:18.337295 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:18.337368 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:18.337758 (+ 390us) server_negotiation.cc:299] Negotiation successful
0925 02:29:18.337830 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
I20260925 02:29:18.340879 25109 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:29:18.342823 25109 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:29:18.351030 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.345839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:33471 (local address 127.0.0.1:48246)
0925 02:29:18.346511 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:18.346519 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:18.346562 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:18.346893 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:18.346895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:18.346897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:18.346961 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:18.346962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.348184 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.348185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:18.349495 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:18.349498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.350613 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.350614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:18.350778 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.350792 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 02:29:18.350863 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 02:29:18.350956 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":64,"threads_started":1}
I20260925 02:29:18.351624 25617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.345929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48246 (local address 127.11.220.193:33471)
0925 02:29:18.346061 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:18.346066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:18.346640 (+ 574us) server_negotiation.cc:407] Connection header received
0925 02:29:18.346706 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:18.346709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:18.346726 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:18.346795 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:18.347097 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.348078 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.349608 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.350440 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.350515 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.350574 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:18.351508 (+ 934us) server_negotiation.cc:299] Negotiation successful
0925 02:29:18.351549 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260925 02:29:18.352432 25242 tablet_service.cc:1467] Processing CreateTablet for tablet 94c30ce9542345c7ab8e42666e07ab1b (DEFAULT_TABLE table=test-table [id=1566c45b1bca462e9aa9abf98e9d2bf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:18.352882 25242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94c30ce9542345c7ab8e42666e07ab1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:18.355057 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.346006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:38923 (local address 127.0.0.1:57030)
0925 02:29:18.351379 (+ 5373us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:18.351389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:18.351434 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:18.351672 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:18.351673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:18.351675 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:18.351702 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:18.351703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.352826 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.352827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:18.353868 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:18.353871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.354851 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.354852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:18.354910 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.354912 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:29:18.354954 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 02:29:18.354998 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5364,"thread_start_us":56,"threads_started":1}
I20260925 02:29:18.355314 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.346193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57030 (local address 127.11.220.194:38923)
0925 02:29:18.350988 (+ 4795us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:18.350993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:18.351445 (+ 452us) server_negotiation.cc:407] Connection header received
0925 02:29:18.351484 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:18.351486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:18.351502 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:18.351588 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:18.351764 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.352737 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.353979 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.354705 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.354774 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.354821 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:18.355216 (+ 395us) server_negotiation.cc:299] Negotiation successful
0925 02:29:18.355250 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4766,"thread_start_us":57,"threads_started":1}
I20260925 02:29:18.355981 25376 tablet_service.cc:1467] Processing CreateTablet for tablet 94c30ce9542345c7ab8e42666e07ab1b (DEFAULT_TABLE table=test-table [id=1566c45b1bca462e9aa9abf98e9d2bf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:18.356309 25376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94c30ce9542345c7ab8e42666e07ab1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:18.357295 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.345630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:44347 (local address 127.0.0.1:48744)
0925 02:29:18.346229 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:18.346266 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:18.346326 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:18.352841 (+ 6515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:18.352842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:18.352844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:18.352885 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:18.352886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.354512 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.354513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:18.355527 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:18.355530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.357099 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.357100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:18.357158 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.357161 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:29:18.357203 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 02:29:18.357237 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":72,"threads_started":1}
I20260925 02:29:18.357925 25615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.345706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48744 (local address 127.11.220.195:44347)
0925 02:29:18.352617 (+ 6911us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:18.352621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:18.352632 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:29:18.352721 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:18.352723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:18.352738 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:18.353025 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:18.353104 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.354414 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.355879 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.356764 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.357740 (+ 976us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.357798 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:18.357824 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 02:29:18.357859 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6879,"thread_start_us":51,"threads_started":1}
I20260925 02:29:18.358613 25500 tablet_service.cc:1467] Processing CreateTablet for tablet 94c30ce9542345c7ab8e42666e07ab1b (DEFAULT_TABLE table=test-table [id=1566c45b1bca462e9aa9abf98e9d2bf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:18.358909 25500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94c30ce9542345c7ab8e42666e07ab1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:18.359380 25621 tablet_bootstrap.cc:492] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: Bootstrap starting.
I20260925 02:29:18.359601 25620 tablet_bootstrap.cc:492] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923: Bootstrap starting.
I20260925 02:29:18.360090 25621 tablet_bootstrap.cc:654] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:18.360193 25620 tablet_bootstrap.cc:654] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:18.360513 25620 log.cc:824] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:18.360515 25621 log.cc:824] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:18.361297 25620 tablet_bootstrap.cc:492] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923: No bootstrap required, opened a new log
I20260925 02:29:18.361358 25620 ts_tablet_manager.cc:1397] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:29:18.361418 25621 tablet_bootstrap.cc:492] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: No bootstrap required, opened a new log
I20260925 02:29:18.361466 25621 ts_tablet_manager.cc:1397] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 02:29:18.363737 25620 raft_consensus.cc:348] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 } } peers { permanent_uuid: "5278172fee7e4850851481a8125c8923" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33471 } } peers { permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 } }
I20260925 02:29:18.363737 25621 raft_consensus.cc:348] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 } } peers { permanent_uuid: "5278172fee7e4850851481a8125c8923" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33471 } } peers { permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 } }
I20260925 02:29:18.363893 25621 raft_consensus.cc:374] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:18.363893 25620 raft_consensus.cc:374] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:18.363925 25620 raft_consensus.cc:729] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5278172fee7e4850851481a8125c8923, State: Initialized, Role: FOLLOWER
I20260925 02:29:18.363925 25621 raft_consensus.cc:729] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f95da849282f4926a4738e2d9b334694, State: Initialized, Role: FOLLOWER
I20260925 02:29:18.364050 25621 consensus_queue.cc:260] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 } } peers { permanent_uuid: "5278172fee7e4850851481a8125c8923" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33471 } } peers { permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 } }
I20260925 02:29:18.364310 25621 ts_tablet_manager.cc:1428] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 02:29:18.364418 25449 heartbeater.cc:503] Master 127.11.220.254:36483 was elected leader, sending a full tablet report...
I20260925 02:29:18.364625 25620 consensus_queue.cc:260] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 } } peers { permanent_uuid: "5278172fee7e4850851481a8125c8923" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33471 } } peers { permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 } }
I20260925 02:29:18.364949 25620 ts_tablet_manager.cc:1428] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260925 02:29:18.365106 25310 heartbeater.cc:503] Master 127.11.220.254:36483 was elected leader, sending a full tablet report...
I20260925 02:29:18.365710 25624 tablet_bootstrap.cc:492] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: Bootstrap starting.
I20260925 02:29:18.366173 25624 tablet_bootstrap.cc:654] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:18.366436 25624 log.cc:824] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:18.367203 25624 tablet_bootstrap.cc:492] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: No bootstrap required, opened a new log
I20260925 02:29:18.367291 25624 ts_tablet_manager.cc:1397] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:29:18.369270 25624 raft_consensus.cc:348] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 } } peers { permanent_uuid: "5278172fee7e4850851481a8125c8923" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33471 } } peers { permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 } }
I20260925 02:29:18.369410 25624 raft_consensus.cc:374] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:18.369443 25624 raft_consensus.cc:729] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72da224d63df4dd581704ca71adb1f89, State: Initialized, Role: FOLLOWER
I20260925 02:29:18.369556 25624 consensus_queue.cc:260] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 } } peers { permanent_uuid: "5278172fee7e4850851481a8125c8923" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33471 } } peers { permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 } }
I20260925 02:29:18.369783 25624 ts_tablet_manager.cc:1428] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:29:18.369877 25587 heartbeater.cc:503] Master 127.11.220.254:36483 was elected leader, sending a full tablet report...
I20260925 02:29:18.385790 25626 raft_consensus.cc:482] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:29:18.386268 25626 raft_consensus.cc:504] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 } } peers { permanent_uuid: "5278172fee7e4850851481a8125c8923" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33471 } } peers { permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 } }
I20260925 02:29:18.387303 25626 leader_election.cc:288] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72da224d63df4dd581704ca71adb1f89 (127.11.220.195:44347), f95da849282f4926a4738e2d9b334694 (127.11.220.194:38923)
I20260925 02:29:18.391067 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.387251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:55555 (local address 127.11.220.194:38923)
0925 02:29:18.387286 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:18.387288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:18.387676 (+ 388us) server_negotiation.cc:407] Connection header received
0925 02:29:18.387762 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:18.387763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:18.387771 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:18.387797 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:18.387965 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.388643 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.390063 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.390736 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.390779 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.390787 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:18.390977 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 02:29:18.391008 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:29:18.391425 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.387088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:38923 (local address 127.11.220.193:55555)
0925 02:29:18.387630 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:18.387638 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:18.387697 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:18.387851 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:18.387851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:18.387853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:18.387899 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:18.387900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.388737 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.388738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:18.389975 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:18.389978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.390852 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.390853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:18.390912 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.390915 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:29:18.391290 (+ 375us) client_negotiation.cc:240] Negotiation successful
0925 02:29:18.391325 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":70,"threads_started":1}
I20260925 02:29:18.391836 25402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c30ce9542345c7ab8e42666e07ab1b" candidate_uuid: "5278172fee7e4850851481a8125c8923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95da849282f4926a4738e2d9b334694" is_pre_election: true
I20260925 02:29:18.391961 25402 raft_consensus.cc:2454] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5278172fee7e4850851481a8125c8923 in term 0.
I20260925 02:29:18.392295 25196 leader_election.cc:302] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5278172fee7e4850851481a8125c8923, f95da849282f4926a4738e2d9b334694; no voters:
I20260925 02:29:18.392328 25615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.386951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:36915 (local address 127.11.220.195:44347)
0925 02:29:18.387002 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:18.387004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:18.387525 (+ 521us) server_negotiation.cc:407] Connection header received
0925 02:29:18.387595 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:18.387595 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:18.387606 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:18.387656 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:18.389656 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.390270 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.391462 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.391968 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.392012 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.392020 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:18.392224 (+ 204us) server_negotiation.cc:299] Negotiation successful
0925 02:29:18.392262 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:29:18.392529 25630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.386775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:44347 (local address 127.11.220.193:36915)
0925 02:29:18.387464 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:18.387475 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:18.387538 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:18.387761 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:18.387761 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:18.387763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:18.387835 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:18.387836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.390374 (+ 2538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.390375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:18.391370 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:18.391373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.392077 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.392077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:18.392145 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.392148 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:29:18.392406 (+ 258us) client_negotiation.cc:240] Negotiation successful
0925 02:29:18.392469 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":96,"threads_started":1}
I20260925 02:29:18.392563 25626 raft_consensus.cc:2781] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:29:18.392613 25626 raft_consensus.cc:482] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:29:18.392639 25626 raft_consensus.cc:3037] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:18.392793 25540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c30ce9542345c7ab8e42666e07ab1b" candidate_uuid: "5278172fee7e4850851481a8125c8923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72da224d63df4dd581704ca71adb1f89" is_pre_election: true
I20260925 02:29:18.392896 25540 raft_consensus.cc:2454] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5278172fee7e4850851481a8125c8923 in term 0.
I20260925 02:29:18.393477 25626 raft_consensus.cc:504] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 } } peers { permanent_uuid: "5278172fee7e4850851481a8125c8923" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33471 } } peers { permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 } }
I20260925 02:29:18.393656 25626 leader_election.cc:288] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [CANDIDATE]: Term 1 election: Requested vote from peers 72da224d63df4dd581704ca71adb1f89 (127.11.220.195:44347), f95da849282f4926a4738e2d9b334694 (127.11.220.194:38923)
I20260925 02:29:18.393714 25540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c30ce9542345c7ab8e42666e07ab1b" candidate_uuid: "5278172fee7e4850851481a8125c8923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72da224d63df4dd581704ca71adb1f89"
I20260925 02:29:18.393788 25540 raft_consensus.cc:3037] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:18.393788 25402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94c30ce9542345c7ab8e42666e07ab1b" candidate_uuid: "5278172fee7e4850851481a8125c8923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f95da849282f4926a4738e2d9b334694"
I20260925 02:29:18.393844 25402 raft_consensus.cc:3037] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:18.394588 25540 raft_consensus.cc:2454] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5278172fee7e4850851481a8125c8923 in term 1.
I20260925 02:29:18.394587 25402 raft_consensus.cc:2454] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5278172fee7e4850851481a8125c8923 in term 1.
I20260925 02:29:18.394757 25196 leader_election.cc:302] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5278172fee7e4850851481a8125c8923, f95da849282f4926a4738e2d9b334694; no voters:
I20260925 02:29:18.394833 25626 raft_consensus.cc:2781] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:29:18.394944 25626 raft_consensus.cc:686] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [term 1 LEADER]: Becoming Leader. State: Replica: 5278172fee7e4850851481a8125c8923, State: Running, Role: LEADER
I20260925 02:29:18.395027 25626 consensus_queue.cc:237] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 } } peers { permanent_uuid: "5278172fee7e4850851481a8125c8923" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33471 } } peers { permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 } }
I20260925 02:29:18.395743 25108 catalog_manager.cc:5495] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5278172fee7e4850851481a8125c8923 (127.11.220.193). New cstate: current_term: 1 leader_uuid: "5278172fee7e4850851481a8125c8923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5278172fee7e4850851481a8125c8923" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 } health_report { overall_health: UNKNOWN } } }
W20260925 02:29:18.403501 25450 tablet.cc:2369] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:29:18.414132 25617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:18.410961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.11.220.193:33471)
0925 02:29:18.410996 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:18.410998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:18.411006 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:29:18.411087 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:18.411088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:18.411122 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:18.411200 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:29:18.411386 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.412063 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.413141 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:18.413428 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:18.413490 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:18.413682 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:29:18.413793 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:29:18.413880 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:18.414049 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 02:29:18.414068 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 25 02:29:18 dist-test-slave-dn4n krb5kdc[24085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303357, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for HTTP/127.11.220.212@KRBTEST.COM
I20260925 02:29:18.416111 25402 raft_consensus.cc:1261] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694 [term 1 FOLLOWER]: Refusing update from remote peer 5278172fee7e4850851481a8125c8923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:29:18.416116 25540 raft_consensus.cc:1261] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89 [term 1 FOLLOWER]: Refusing update from remote peer 5278172fee7e4850851481a8125c8923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:29:18.416412 25626 consensus_queue.cc:1035] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72da224d63df4dd581704ca71adb1f89" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 44347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:29:18.416718 25633 consensus_queue.cc:1035] T 94c30ce9542345c7ab8e42666e07ab1b P 5278172fee7e4850851481a8125c8923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f95da849282f4926a4738e2d9b334694" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 38923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:29:18.422966 25636 mvcc.cc:205] Tried to move back new op lower bound from 7333082556068909056 to 7333082555986513920. Current Snapshot: MvccSnapshot[applied={T|T < 7333082556068909056}]
I20260925 02:29:18.423398 25638 mvcc.cc:205] Tried to move back new op lower bound from 7333082556068909056 to 7333082555986513920. Current Snapshot: MvccSnapshot[applied={T|T < 7333082556068909056}]
I20260925 02:29:18.425400 25637 mvcc.cc:205] Tried to move back new op lower bound from 7333082556068909056 to 7333082555986513920. Current Snapshot: MvccSnapshot[applied={T|T < 7333082556068909056}]
I20260925 02:29:18.429814 25109 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:29:18.429893 25109 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:29:18.430560 25109 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47030303b3b44ed3934bdd8318a66373: Sending DeleteTablet for 3 replicas of tablet 94c30ce9542345c7ab8e42666e07ab1b
I20260925 02:29:18.430989 25500 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c30ce9542345c7ab8e42666e07ab1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:18 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:48744
I20260925 02:29:18.431041 25242 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c30ce9542345c7ab8e42666e07ab1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:18 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:48246
I20260925 02:29:18.431422 25376 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c30ce9542345c7ab8e42666e07ab1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:18 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:57030
I20260925 02:29:18.431926 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 25148
I20260925 02:29:18.436607 25645 tablet_replica.cc:330] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: stopping tablet replica
I20260925 02:29:18.436767 25645 raft_consensus.cc:2229] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:29:18.436918 25645 raft_consensus.cc:2258] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:29:18.437099 25647 tablet_replica.cc:330] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: stopping tablet replica
I20260925 02:29:18.437220 25647 raft_consensus.cc:2229] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:29:18.437353 25647 raft_consensus.cc:2258] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:29:18.437659 25645 ts_tablet_manager.cc:1905] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:29:18.437836 25647 ts_tablet_manager.cc:1905] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:29:18.439447 25645 ts_tablet_manager.cc:1918] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:29:18.439576 25645 log.cc:1196] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/94c30ce9542345c7ab8e42666e07ab1b
I20260925 02:29:18.439899 25645 ts_tablet_manager.cc:1939] T 94c30ce9542345c7ab8e42666e07ab1b P 72da224d63df4dd581704ca71adb1f89: Deleting consensus metadata
I20260925 02:29:18.440369 25094 catalog_manager.cc:4841] TS 72da224d63df4dd581704ca71adb1f89 (127.11.220.195:44347): tablet 94c30ce9542345c7ab8e42666e07ab1b (table test-table [id=1566c45b1bca462e9aa9abf98e9d2bf9]) successfully deleted
I20260925 02:29:18.440584 25647 ts_tablet_manager.cc:1918] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:29:18.440923 25647 log.cc:1196] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/94c30ce9542345c7ab8e42666e07ab1b
I20260925 02:29:18.441210 25647 ts_tablet_manager.cc:1939] T 94c30ce9542345c7ab8e42666e07ab1b P f95da849282f4926a4738e2d9b334694: Deleting consensus metadata
I20260925 02:29:18.441602 25093 catalog_manager.cc:4841] TS f95da849282f4926a4738e2d9b334694 (127.11.220.194:38923): tablet 94c30ce9542345c7ab8e42666e07ab1b (table test-table [id=1566c45b1bca462e9aa9abf98e9d2bf9]) successfully deleted
W20260925 02:29:18.446866 25094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:33471 (error 108)
W20260925 02:29:18.447683 25618 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:29:18.447289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:33471 (local address 127.0.0.1:48260)
0925 02:29:18.447345 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:18.447356 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:18.447394 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:18.447641 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:33471: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 02:29:18.447795 25094 catalog_manager.cc:4584] TS 5278172fee7e4850851481a8125c8923 (127.11.220.193:33471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94c30ce9542345c7ab8e42666e07ab1b: Network error: Client connection negotiation failed: client connection to 127.11.220.193:33471: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 02:29:18.448478 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 25314
I20260925 02:29:18.456674 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 25452
I20260925 02:29:18.461709 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 25075
I20260925 02:29:18.472841 12147 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 02:29:18.817296 12147 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 02:29:18.817371 12147 mini_ranger.cc:66] Stopping Ranger...
I20260925 02:29:19.161801 12147 mini_ranger.cc:68] Stopped Ranger
2026-09-25 02:29:19.162 UTC [24117] LOG: received smart shutdown request
2026-09-25 02:29:19.166 UTC [24117] LOG: background worker "logical replication launcher" (PID 24127) exited with exit code 1
2026-09-25 02:29:19.166 UTC [24121] LOG: shutting down
2026-09-25 02:29:19.306 UTC [24117] LOG: database system is shut down
2026-09-25T02:29:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (43040 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:29:19 dist-test-slave-dn4n krb5kdc[25659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:29:19 dist-test-slave-dn4n krb5kdc[25659](info): set up 2 sockets
Sep 25 02:29:19 dist-test-slave-dn4n krb5kdc[25659](info): commencing operation
krb5kdc: starting...
W20260925 02:29:20.356987 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:29:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:29:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:29:20.458452 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:42205
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:42205,127.11.220.253:46227,127.11.220.252:45945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:20.543396 25672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:20.543588 25672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:20.543622 25672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:20.544713 25672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:29:20.544765 25672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:20.544788 25672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:20.544816 25672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:29:20.544839 25672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:29:20.546139 25672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:42205,127.11.220.253:46227,127.11.220.252:45945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:42205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:20.546373 25672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:20.546566 25672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:20.551949 25678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:20.551967 25680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:20.551963 25677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:20.552349 25672 server_base.cc:1034] running on GCE node
I20260925 02:29:20.552582 25672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:20.552874 25672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:20.554020 25672 hybrid_clock.cc:648] HybridClock initialized: now 1790303360554013 us; error 39 us; skew 500 ppm
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:20.556561 25672 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:29:20.557018 25672 webserver.cc:460] Webserver started at http://127.11.220.254:46515/ using document root <none> and password file <none>
I20260925 02:29:20.557255 25672 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:20.557343 25672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:20.557461 25672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:20.558696 25672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "75dccfbf1251453bb00393a46bb8dba6"
format_stamp: "Formatted at 2026-09-25 02:29:20 on dist-test-slave-dn4n"
server_key: "ab310ff08298260c334407773af0ae3f"
server_key_iv: "8279f6b79b4c9f9c97b1a9ef8ba710db"
server_key_version: "encryptionkey@0"
I20260925 02:29:20.559092 25672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "75dccfbf1251453bb00393a46bb8dba6"
format_stamp: "Formatted at 2026-09-25 02:29:20 on dist-test-slave-dn4n"
server_key: "ab310ff08298260c334407773af0ae3f"
server_key_iv: "8279f6b79b4c9f9c97b1a9ef8ba710db"
server_key_version: "encryptionkey@0"
I20260925 02:29:20.560834 25672 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260925 02:29:20.561795 25687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:20.561960 25672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:20.562049 25672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "75dccfbf1251453bb00393a46bb8dba6"
format_stamp: "Formatted at 2026-09-25 02:29:20 on dist-test-slave-dn4n"
server_key: "ab310ff08298260c334407773af0ae3f"
server_key_iv: "8279f6b79b4c9f9c97b1a9ef8ba710db"
server_key_version: "encryptionkey@0"
I20260925 02:29:20.562117 25672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:20.580706 25672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:20.582621 25672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:20.582724 25672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:20.586398 25672 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:42205
I20260925 02:29:20.586448 25740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:42205 every 8 connection(s)
I20260925 02:29:20.586751 25672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:29:20.587502 25741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:20.588896 25741 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:29:20.592600 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 25672
I20260925 02:29:20.592705 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:29:20.592875 12147 external_mini_cluster.cc:1428] Setting key 811b25daa8b20c26196e2d5d10da8415
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:20.597792 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.589499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:42205 (local address 127.0.0.1:53210)
0925 02:29:20.589655 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.589689 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:20.590085 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:20.590302 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:20.590304 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:20.590511 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:20.590652 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.590655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.591531 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.591531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.592161 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.592163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.592659 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.592660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.592721 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.593175 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:20.593183 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:20.594406 (+ 1223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:20.596376 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.596384 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.596386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.596642 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.596901 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.596901 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.596902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.596982 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.597407 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:20.597413 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:20.597467 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 02:29:20.597539 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:29:20.597657 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260925 02:29:20.597913 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.589505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53210 (local address 127.11.220.254:42205)
0925 02:29:20.589651 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.589659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.589772 (+ 113us) server_negotiation.cc:407] Connection header received
0925 02:29:20.590203 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.590204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.590220 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.590296 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.590724 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.591454 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.592228 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.592573 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.592686 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.594499 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.594505 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.594505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.594511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.596286 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.596721 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.596724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.596725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.596826 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.597049 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.597050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.597050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.597331 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.597420 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.597597 (+ 177us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.597742 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260925 02:29:20.598486 25741 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:29:20.598799 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.598689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:46227 (local address 127.0.0.1:38576)
0925 02:29:20.598733 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.598772 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:46227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 02:29:20.598932 25688 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.220.253:46227: connect: Connection refused (error 111)
I20260925 02:29:20.599282 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.599190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:46227 (local address 127.0.0.1:38588)
0925 02:29:20.599239 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.599261 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:46227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 02:29:20.599378 25741 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.253:46227: Network error: Client connection negotiation failed: client connection to 127.11.220.253:46227: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.220.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab.
Entry for principal kudu/127.11.220.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab.
I20260925 02:29:20.615482 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.253:46227
--webserver_interface=127.11.220.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab
--principal=kudu/127.11.220.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:42205,127.11.220.253:46227,127.11.220.252:45945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:29:20.648581 25741 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } attempt: 1
I20260925 02:29:20.648919 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.648816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:46227 (local address 127.0.0.1:38598)
0925 02:29:20.648870 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.648898 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:46227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 02:29:20.649233 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.649121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:46227 (local address 127.0.0.1:38602)
0925 02:29:20.649160 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.649198 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:46227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260925 02:29:20.649300 25741 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.253:46227: Network error: Client connection negotiation failed: client connection to 127.11.220.253:46227: connect: Connection refused (error 111)
W20260925 02:29:20.688023 25748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:20.688192 25748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:20.688213 25748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:20.689311 25748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:29:20.689345 25748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:20.689357 25748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:20.689402 25748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:29:20.689426 25748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:29:20.690742 25748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:42205,127.11.220.253:46227,127.11.220.252:45945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab
--principal=kudu/127.11.220.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.253:46227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.220.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:20.690958 25748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:20.691162 25748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:20.695523 25756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:20.695524 25754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:20.695539 25753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:20.695732 25748 server_base.cc:1034] running on GCE node
I20260925 02:29:20.696027 25748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:20.696360 25748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:20.697499 25748 hybrid_clock.cc:648] HybridClock initialized: now 1790303360697493 us; error 38 us; skew 500 ppm
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:20.705799 25748 init.cc:377] Logged in from keytab as kudu/127.11.220.253@KRBTEST.COM (short username kudu)
I20260925 02:29:20.706184 25748 webserver.cc:460] Webserver started at http://127.11.220.253:36709/ using document root <none> and password file <none>
I20260925 02:29:20.706383 25748 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:20.706442 25748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:20.706585 25748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:20.707510 25748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/data/instance:
uuid: "7745ee55ed41425294f563dd462c70b2"
format_stamp: "Formatted at 2026-09-25 02:29:20 on dist-test-slave-dn4n"
server_key: "247e57077a4ebfdebf36bfe82466a035"
server_key_iv: "8f50afaacad6ca26ce38075285b21a27"
server_key_version: "encryptionkey@0"
I20260925 02:29:20.707837 25748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/wal/instance:
uuid: "7745ee55ed41425294f563dd462c70b2"
format_stamp: "Formatted at 2026-09-25 02:29:20 on dist-test-slave-dn4n"
server_key: "247e57077a4ebfdebf36bfe82466a035"
server_key_iv: "8f50afaacad6ca26ce38075285b21a27"
server_key_version: "encryptionkey@0"
I20260925 02:29:20.709131 25748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:29:20.709781 25764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:20.709935 25748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:20.710024 25748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/wal
uuid: "7745ee55ed41425294f563dd462c70b2"
format_stamp: "Formatted at 2026-09-25 02:29:20 on dist-test-slave-dn4n"
server_key: "247e57077a4ebfdebf36bfe82466a035"
server_key_iv: "8f50afaacad6ca26ce38075285b21a27"
server_key_version: "encryptionkey@0"
I20260925 02:29:20.710107 25748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:20.717425 25741 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } attempt: 2
I20260925 02:29:20.717788 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.717672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:46227 (local address 127.0.0.1:38610)
0925 02:29:20.717730 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.717762 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:46227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 02:29:20.718057 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.717986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:46227 (local address 127.0.0.1:38614)
0925 02:29:20.718021 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.718042 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:46227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260925 02:29:20.718173 25741 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.253:46227: Network error: Client connection negotiation failed: client connection to 127.11.220.253:46227: connect: Connection refused (error 111)
I20260925 02:29:20.723160 25748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:20.724763 25748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:20.724905 25748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:20.728310 25748 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.253:46227
I20260925 02:29:20.728386 25817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.253:46227 every 8 connection(s)
I20260925 02:29:20.728667 25748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
I20260925 02:29:20.729418 25818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:20.729456 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 25748
I20260925 02:29:20.729535 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-1/wal/instance
I20260925 02:29:20.729732 12147 external_mini_cluster.cc:1428] Setting key 0e547d2d506495f4951c95c20e4c8a1f
I20260925 02:29:20.731501 25818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.220.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.252@KRBTEST.COM" created.
I20260925 02:29:20.740525 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.732089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:42205 (local address 127.0.0.1:53216)
0925 02:29:20.732227 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.732263 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:20.732734 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:20.732943 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:20.732946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:20.733148 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:20.733364 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.733369 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.734309 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.734310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.735023 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.735025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.735524 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.735524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.735611 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.736065 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:20.736072 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:20.737553 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:20.739178 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.739185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.739187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.739419 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.739598 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.739598 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.739599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.739719 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.740040 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:20.740045 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:20.740151 (+ 106us) client_negotiation.cc:769] Sending connection context
0925 02:29:20.740229 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 02:29:20.740343 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260925 02:29:20.740545 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.732120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53216 (local address 127.11.220.254:42205)
0925 02:29:20.732185 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.732187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.732338 (+ 151us) server_negotiation.cc:407] Connection header received
0925 02:29:20.732817 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.732819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.732832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.732899 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.733453 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.734220 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.735110 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.735435 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.735506 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.737643 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.737644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.737645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.737651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.739096 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.739482 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.739483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.739483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.739532 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.739795 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.739796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.739796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.739959 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.740043 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.740295 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.740384 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:29:20.741164 25818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
I20260925 02:29:20.747679 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.741418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:46227 (local address 127.0.0.1:38618)
0925 02:29:20.741667 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.741677 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:20.741721 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:20.741858 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:20.741858 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:20.741988 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:20.742025 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.742026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.742828 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.742829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.743270 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.743271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.743699 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.743699 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.743757 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.744144 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:20.744149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:20.745472 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:20.746895 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.746896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.746896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.747086 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.747235 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.747236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.747236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.747280 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.747457 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:20.747458 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:20.747506 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 02:29:20.747568 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 02:29:20.747627 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239}
I20260925 02:29:20.747869 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.741525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38618 (local address 127.11.220.253:46227)
0925 02:29:20.741666 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.741671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.741721 (+ 50us) server_negotiation.cc:407] Connection header received
0925 02:29:20.741776 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.741778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.741790 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.741856 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.742109 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.742748 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.743355 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.743618 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.743697 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.745553 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.745559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.745559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.745564 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.746826 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.747143 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.747145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.747145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.747186 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.747321 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.747321 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.747321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.747414 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.747463 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.747666 (+ 203us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.747758 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260925 02:29:20.748420 25818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:29:20.748746 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.748632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37668)
0925 02:29:20.748679 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.748717 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 02:29:20.748862 25765 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
I20260925 02:29:20.749199 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.749123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37680)
0925 02:29:20.749163 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.749185 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 02:29:20.749275 25818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:45945: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab.
Entry for principal kudu/127.11.220.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab.
I20260925 02:29:20.752514 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.252:45945
--webserver_interface=127.11.220.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab
--principal=kudu/127.11.220.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:42205,127.11.220.253:46227,127.11.220.252:45945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:29:20.798436 25818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } attempt: 1
I20260925 02:29:20.798801 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.798689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37684)
0925 02:29:20.798742 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.798775 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 02:29:20.799115 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.799030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37700)
0925 02:29:20.799072 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.799096 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 02:29:20.799234 25818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:45945: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
I20260925 02:29:20.809321 25741 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } attempt: 3
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
I20260925 02:29:20.815824 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.809626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38626 (local address 127.11.220.253:46227)
0925 02:29:20.809694 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.809696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.809705 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:29:20.809757 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.809758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.809766 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.809824 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.810133 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.810575 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.811202 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.811396 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.811457 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.813473 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.813474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.813475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.813480 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.814725 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.815098 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.815099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.815099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.815147 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.815324 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.815325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.815325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.815413 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.815485 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.815630 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.815706 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 02:29:20.815827 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.809555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:46227 (local address 127.0.0.1:38626)
0925 02:29:20.809608 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.809623 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:20.809698 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:20.809830 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:20.809831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:20.809988 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:20.810044 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.810045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.810654 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.810655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.811113 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.811115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.811467 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.811468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.811542 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.812021 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:20.812025 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:20.813395 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:20.814816 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.814817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.814818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.815029 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.815204 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.815205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.815205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.815267 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.815482 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:20.815483 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:20.815527 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 02:29:20.815608 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 02:29:20.815704 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 02:29:20.816192 25741 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:29:20.816525 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.816403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37708)
0925 02:29:20.816475 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.816506 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 02:29:20.816809 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.816718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37724)
0925 02:29:20.816767 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.816790 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 02:29:20.816898 25741 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:45945: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
W20260925 02:29:20.827135 25825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:20.827327 25825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:20.827369 25825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:20.828557 25825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:29:20.828615 25825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:20.828640 25825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:20.828666 25825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:29:20.828692 25825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:29:20.830193 25825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:42205,127.11.220.253:46227,127.11.220.252:45945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab
--principal=kudu/127.11.220.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.252:45945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.220.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:20.830448 25825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:20.830638 25825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:20.834919 25830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:20.834954 25825 server_base.cc:1034] running on GCE node
W20260925 02:29:20.834952 25831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:20.834921 25833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:20.835356 25825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:20.835565 25825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:20.836689 25825 hybrid_clock.cc:648] HybridClock initialized: now 1790303360836680 us; error 23 us; skew 500 ppm
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:20.838783 25825 init.cc:377] Logged in from keytab as kudu/127.11.220.252@KRBTEST.COM (short username kudu)
I20260925 02:29:20.839198 25825 webserver.cc:460] Webserver started at http://127.11.220.252:42103/ using document root <none> and password file <none>
I20260925 02:29:20.839398 25825 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:20.839458 25825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:20.839594 25825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:20.840605 25825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/data/instance:
uuid: "159bc4d0695b4114a33a3f2e55ec505b"
format_stamp: "Formatted at 2026-09-25 02:29:20 on dist-test-slave-dn4n"
server_key: "db0f5e0fa7fadab0a328bad3dbb94024"
server_key_iv: "5b3667aea62a241f27e1d18366b869ea"
server_key_version: "encryptionkey@0"
I20260925 02:29:20.840932 25825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/wal/instance:
uuid: "159bc4d0695b4114a33a3f2e55ec505b"
format_stamp: "Formatted at 2026-09-25 02:29:20 on dist-test-slave-dn4n"
server_key: "db0f5e0fa7fadab0a328bad3dbb94024"
server_key_iv: "5b3667aea62a241f27e1d18366b869ea"
server_key_version: "encryptionkey@0"
I20260925 02:29:20.842096 25825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:29:20.842694 25840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:20.842856 25825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:29:20.842945 25825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/wal
uuid: "159bc4d0695b4114a33a3f2e55ec505b"
format_stamp: "Formatted at 2026-09-25 02:29:20 on dist-test-slave-dn4n"
server_key: "db0f5e0fa7fadab0a328bad3dbb94024"
server_key_iv: "5b3667aea62a241f27e1d18366b869ea"
server_key_version: "encryptionkey@0"
I20260925 02:29:20.843014 25825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:20.848148 25741 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } attempt: 1
I20260925 02:29:20.848515 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.848376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37740)
0925 02:29:20.848430 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.848481 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 02:29:20.848789 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.848705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37750)
0925 02:29:20.848747 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.848769 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 02:29:20.848877 25741 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:45945: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
I20260925 02:29:20.866966 25825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:20.867355 25818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } attempt: 2
I20260925 02:29:20.867683 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.867569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37758)
0925 02:29:20.867623 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.867654 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 02:29:20.868048 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.867907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37768)
0925 02:29:20.867970 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.867999 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 02:29:20.868162 25818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:45945: Network error: Client connection negotiation failed: client connection to 127.11.220.252:45945: connect: Connection refused (error 111)
I20260925 02:29:20.868976 25825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:20.869123 25825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:20.872610 25825 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.252:45945
I20260925 02:29:20.872658 25893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.252:45945 every 8 connection(s)
I20260925 02:29:20.872963 25825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
I20260925 02:29:20.873730 25894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:20.875003 25894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:29:20.876252 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 25825
I20260925 02:29:20.876353 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790303222243611-12147-0/minicluster-data/master-2/wal/instance
I20260925 02:29:20.876554 12147 external_mini_cluster.cc:1428] Setting key f12574258dd0f09a890290f9f1936a0e
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
I20260925 02:29:20.883476 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.875573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53218 (local address 127.11.220.254:42205)
0925 02:29:20.875607 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.875609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.875783 (+ 174us) server_negotiation.cc:407] Connection header received
0925 02:29:20.876200 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.876201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.876210 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.876246 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.877013 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.877740 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.878618 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.878817 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.878894 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.880857 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.880860 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.880860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.880870 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.882171 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.882537 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.882539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.882539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.882596 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.882887 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.882888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.882889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.883025 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.883087 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.883367 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.883424 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:29:20.883673 25896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.875557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:42205 (local address 127.0.0.1:53218)
0925 02:29:20.875678 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.875711 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:20.876112 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:20.876311 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:20.876316 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:20.876639 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:20.876892 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.876896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.877855 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.877855 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.878523 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.878525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.878906 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.878907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.879038 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.879422 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:20.879430 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:20.880767 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:20.882251 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.882257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.882258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.882466 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.882677 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.882678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.882679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.882817 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.883093 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:20.883099 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:20.883237 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 02:29:20.883344 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 02:29:20.883479 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":72,"threads_started":1}
I20260925 02:29:20.884272 25894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:29:20.885030 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.876889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37778 (local address 127.11.220.252:45945)
0925 02:29:20.877016 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.877020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.877041 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:29:20.877108 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.877109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.877124 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.877175 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.877848 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.878526 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.879789 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.880049 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.880153 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.882166 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.882174 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.882174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.882180 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.883839 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.884227 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.884229 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.884230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.884285 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.884467 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.884468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.884468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.884614 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.884740 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.884817 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.884915 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":102,"thread_start_us":81,"threads_started":1}
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:20.890166 25896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.884497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:46227 (local address 127.0.0.1:38630)
0925 02:29:20.884572 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.884582 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:20.884621 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:20.884742 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:20.884743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:20.884897 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:20.884941 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.884942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.885592 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.885593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.886143 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.886145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.886448 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.886449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.886510 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.886829 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:20.886834 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:20.887812 (+ 978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:20.889253 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.889254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.889254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.889404 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.889606 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.889606 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.889606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.889651 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.889909 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:20.889909 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:20.889950 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 02:29:20.890023 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:29:20.890096 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 02:29:20.890239 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.884539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38630 (local address 127.11.220.253:46227)
0925 02:29:20.884576 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.884578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.884609 (+ 31us) server_negotiation.cc:407] Connection header received
0925 02:29:20.884682 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.884682 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.884690 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.884739 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.885016 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.885531 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.886224 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.886386 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.886431 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.887882 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.887883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.887884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.887889 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.889178 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.889475 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.889477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.889477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.889530 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.889708 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.889709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.889709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.889841 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.889890 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.890068 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.890144 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:29:20.890547 25894 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:29:20.892386 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.885642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53224 (local address 127.11.220.254:42205)
0925 02:29:20.885674 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.885675 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.885681 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:29:20.885722 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.885723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.885730 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.885806 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.886162 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.886689 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.887293 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.887474 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.887535 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.889826 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.889827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.889827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.889832 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.891395 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.891700 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.891701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.891701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.891748 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.891930 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.891931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.891931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.892028 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.892209 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.892243 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.892308 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
I20260925 02:29:20.896888 25896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.890783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37788)
0925 02:29:20.891025 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.891035 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:20.891077 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:20.891185 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:20.891185 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:20.891302 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:20.891322 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.891323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.891848 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.891849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.892394 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.892396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.892925 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.892926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.893000 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.893358 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:20.893362 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:20.894371 (+ 1009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:20.895910 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.895911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.895911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.896055 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.896262 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.896263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.896263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.896355 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.896676 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:20.896676 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:20.896708 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 02:29:20.896759 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:29:20.896836 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229}
I20260925 02:29:20.897091 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.890814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37788 (local address 127.11.220.252:45945)
0925 02:29:20.890846 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.890847 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.891075 (+ 228us) server_negotiation.cc:407] Connection header received
0925 02:29:20.891126 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.891127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.891135 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.891182 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.891393 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.891781 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.892597 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.892856 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.892916 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.894436 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.894437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.894437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.894444 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.895836 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.896128 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.896129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.896130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.896186 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.896487 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.896487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.896488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.896612 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.896676 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.896829 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.896907 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
I20260925 02:29:20.898337 25894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b: Bootstrap starting.
I20260925 02:29:20.899081 25894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:20.899366 25894 log.cc:824] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:20.900185 25894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b: No bootstrap required, opened a new log
I20260925 02:29:20.900301 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.892973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38646 (local address 127.11.220.253:46227)
0925 02:29:20.893013 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.893015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.893022 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:29:20.893039 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.893040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.893048 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.893365 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.893623 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.894256 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.895100 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.895360 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.895410 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.897861 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.897862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.897863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.897868 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.899295 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.899631 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.899631 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.899632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.899691 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.899854 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.899855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.899855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.899955 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.900078 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.900152 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.900201 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:29:20.901747 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:29:20.901924 25894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } }
I20260925 02:29:20.902050 25894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:20.902096 25894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 159bc4d0695b4114a33a3f2e55ec505b, State: Initialized, Role: FOLLOWER
I20260925 02:29:20.902197 25894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } }
I20260925 02:29:20.902420 25900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } } }
I20260925 02:29:20.902473 25894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:29:20.902489 25900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:29:20.904192 25918 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:29:20.904241 25918 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:29:20.904381 25825 master_runner.cc:186] Error getting master registration for 127.11.220.254:42205: OK, instance_id { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" instance_seqno: 1790303360585404 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:29:20.906581 25825 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:29:20.910557 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.903089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.11.220.254:42205)
0925 02:29:20.903119 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.903121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.903463 (+ 342us) server_negotiation.cc:407] Connection header received
0925 02:29:20.903542 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.903542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.903550 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.903619 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.904250 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.904905 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.906060 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.906360 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.906395 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.907992 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.907993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.907993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.907999 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.909429 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.909782 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.909783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.909783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.909832 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.910022 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.910023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.910023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.910138 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.910199 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.910386 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.910452 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 02:29:20.911237 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.903354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38652 (local address 127.11.220.253:46227)
0925 02:29:20.903389 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.903391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.903650 (+ 259us) server_negotiation.cc:407] Connection header received
0925 02:29:20.903753 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.903754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.903763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.903790 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.904382 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.905113 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.906161 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.906430 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.906466 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.907920 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.907921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.907921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.907927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.909787 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.910398 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.910399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.910399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.910447 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.910665 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.910666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.910666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.910789 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.910857 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.911062 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.911125 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:29:20.913995 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.903298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37792 (local address 127.11.220.252:45945)
0925 02:29:20.903325 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.903327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.906822 (+ 3495us) server_negotiation.cc:407] Connection header received
0925 02:29:20.906922 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.906923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.906932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.906959 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.907876 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.908560 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.909470 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.909756 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.909794 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.911578 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.911579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.911579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.911586 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.912995 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.913269 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.913270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.913270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.913325 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.913530 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.913531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.913531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.913675 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.913761 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.913841 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.913908 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260925 02:29:20.924015 25741 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } attempt: 2
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
I20260925 02:29:20.930001 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.924284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37794)
0925 02:29:20.924332 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.924343 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:20.924401 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:20.924570 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:20.924571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:20.924699 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:20.924727 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.924728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.925287 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.925288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.925722 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.925723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.926028 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.926029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.926083 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.926439 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:20.926444 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:20.927417 (+ 973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:20.929153 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.929154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.929155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.929308 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.929515 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.929515 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.929515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.929573 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.929812 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:20.929812 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:20.929852 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 02:29:20.929897 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 02:29:20.929950 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 02:29:20.930065 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.924353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37794 (local address 127.11.220.252:45945)
0925 02:29:20.924406 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.924408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.924416 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:29:20.924488 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.924489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.924498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.924576 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.924801 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.925204 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.925797 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.925958 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.926019 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.927488 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.927489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.927490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.927507 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.929066 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.929388 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.929389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.929390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.929443 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.929627 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.929628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.929629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.929731 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.929789 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.929934 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.929993 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 02:29:20.931387 25741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6: Bootstrap starting.
I20260925 02:29:20.932060 25741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:20.932328 25741 log.cc:824] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:20.933032 25741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6: No bootstrap required, opened a new log
I20260925 02:29:20.934894 25741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } }
I20260925 02:29:20.935021 25741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:20.935052 25741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75dccfbf1251453bb00393a46bb8dba6, State: Initialized, Role: FOLLOWER
I20260925 02:29:20.935170 25741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } }
I20260925 02:29:20.935520 25741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:29:20.936594 25921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } } }
I20260925 02:29:20.936712 25921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:29:20.937183 25933 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:29:20.937227 25933 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:29:20.937881 25672 master_runner.cc:186] Error getting master registration for 127.11.220.253:46227: OK, instance_id { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" instance_seqno: 1790303360727396 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:29:20.939275 25672 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:29:20.959385 25818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } attempt: 3
Sep 25 02:29:20 dist-test-slave-dn4n krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303360, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
I20260925 02:29:20.965586 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.959802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37808 (local address 127.11.220.252:45945)
0925 02:29:20.959851 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:20.959853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:20.959859 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:29:20.959882 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:20.959883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:20.959890 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:20.959949 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:20.960214 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.960678 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.961267 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.961430 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.961492 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.963100 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:20.963101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:20.963102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:20.963108 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:20.964590 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.964886 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.964887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.964888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.964936 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:20.965101 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:20.965101 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:20.965102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:20.965213 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:20.965281 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:20.965435 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 02:29:20.965497 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 02:29:20.965586 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:20.959647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:45945 (local address 127.0.0.1:37808)
0925 02:29:20.959701 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:20.959709 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:20.959740 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:20.959955 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:20.959956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:20.960099 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:20.960135 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.960136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.960746 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.960747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.961192 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:20.961193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:20.961501 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:20.961502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:20.961554 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:20.961937 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:20.961942 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:20.963004 (+ 1062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:20.964670 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.964671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.964671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.964822 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.964984 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:20.964985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:20.964985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:20.965035 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:20.965275 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:20.965276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:20.965315 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 02:29:20.965411 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 02:29:20.965497 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 02:29:20.966989 25818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2: Bootstrap starting.
I20260925 02:29:20.967571 25818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:20.967846 25818 log.cc:824] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:20.968559 25818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2: No bootstrap required, opened a new log
I20260925 02:29:20.969799 25818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } }
I20260925 02:29:20.969925 25818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:20.969966 25818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7745ee55ed41425294f563dd462c70b2, State: Initialized, Role: FOLLOWER
I20260925 02:29:20.970089 25818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } }
I20260925 02:29:20.970325 25936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } } }
I20260925 02:29:20.970381 25818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:29:20.970445 25936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:29:20.972043 25947 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:29:20.972138 25947 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:29:20.974195 25748 master_runner.cc:263] No leader master found from master 7745ee55ed41425294f563dd462c70b2
I20260925 02:29:20.974247 25748 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 02:29:20.975471 25770 tablet.cc:2369] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:29:21.069542 25936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:29:21.069650 25936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } }
I20260925 02:29:21.069854 25936 leader_election.cc:288] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75dccfbf1251453bb00393a46bb8dba6 (127.11.220.254:42205), 159bc4d0695b4114a33a3f2e55ec505b (127.11.220.252:45945)
I20260925 02:29:21.070331 25714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7745ee55ed41425294f563dd462c70b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75dccfbf1251453bb00393a46bb8dba6" is_pre_election: true
I20260925 02:29:21.070497 25714 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7745ee55ed41425294f563dd462c70b2 in term 0.
I20260925 02:29:21.070883 25767 leader_election.cc:302] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75dccfbf1251453bb00393a46bb8dba6, 7745ee55ed41425294f563dd462c70b2; no voters:
I20260925 02:29:21.071200 25936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:29:21.071264 25936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:29:21.071288 25936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:21.072567 25936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } }
I20260925 02:29:21.072678 25936 leader_election.cc:288] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [CANDIDATE]: Term 1 election: Requested vote from peers 75dccfbf1251453bb00393a46bb8dba6 (127.11.220.254:42205), 159bc4d0695b4114a33a3f2e55ec505b (127.11.220.252:45945)
I20260925 02:29:21.072878 25714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7745ee55ed41425294f563dd462c70b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75dccfbf1251453bb00393a46bb8dba6"
I20260925 02:29:21.072958 25714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:21.073758 25714 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7745ee55ed41425294f563dd462c70b2 in term 1.
I20260925 02:29:21.073915 25767 leader_election.cc:302] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75dccfbf1251453bb00393a46bb8dba6, 7745ee55ed41425294f563dd462c70b2; no voters:
I20260925 02:29:21.074260 25868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7745ee55ed41425294f563dd462c70b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "159bc4d0695b4114a33a3f2e55ec505b" is_pre_election: true
I20260925 02:29:21.074257 25867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7745ee55ed41425294f563dd462c70b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "159bc4d0695b4114a33a3f2e55ec505b"
I20260925 02:29:21.074419 25868 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7745ee55ed41425294f563dd462c70b2 in term 0.
I20260925 02:29:21.075223 25936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:29:21.075376 25936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [term 1 LEADER]: Becoming Leader. State: Replica: 7745ee55ed41425294f563dd462c70b2, State: Running, Role: LEADER
I20260925 02:29:21.075462 25936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } }
I20260925 02:29:21.075822 25949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7745ee55ed41425294f563dd462c70b2. Latest consensus state: current_term: 1 leader_uuid: "7745ee55ed41425294f563dd462c70b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } } }
I20260925 02:29:21.075956 25949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:21.076184 25951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:29:21.076365 25951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:29:21.077817 25868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:21.077865 25714 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [term 1 FOLLOWER]: Refusing update from remote peer 7745ee55ed41425294f563dd462c70b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:29:21.078544 25868 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [term 1 FOLLOWER]: Refusing update from remote peer 7745ee55ed41425294f563dd462c70b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:29:21.078855 25936 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:29:21.079007 25949 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:29:21.080057 25900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7745ee55ed41425294f563dd462c70b2. Latest consensus state: current_term: 1 leader_uuid: "7745ee55ed41425294f563dd462c70b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } } }
I20260925 02:29:21.080132 25900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:29:21.080189 25921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7745ee55ed41425294f563dd462c70b2. Latest consensus state: current_term: 1 leader_uuid: "7745ee55ed41425294f563dd462c70b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } } }
I20260925 02:29:21.080271 25921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:29:21.081137 25936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7745ee55ed41425294f563dd462c70b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } } }
I20260925 02:29:21.081209 25936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:21.081331 25900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7745ee55ed41425294f563dd462c70b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } } }
I20260925 02:29:21.081393 25900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:29:21.082074 25949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7745ee55ed41425294f563dd462c70b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } } }
I20260925 02:29:21.082142 25949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:21.081908 25953 mvcc.cc:205] Tried to move back new op lower bound from 7333082566972702720 to 7333082566965563392. Current Snapshot: MvccSnapshot[applied={T|T < 7333082566972702720}]
I20260925 02:29:21.082567 25921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7745ee55ed41425294f563dd462c70b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75dccfbf1251453bb00393a46bb8dba6" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42205 } } peers { permanent_uuid: "7745ee55ed41425294f563dd462c70b2" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 46227 } } peers { permanent_uuid: "159bc4d0695b4114a33a3f2e55ec505b" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 45945 } } }
I20260925 02:29:21.082636 25921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:29:21.083369 25951 catalog_manager.cc:1347] Generated new cluster ID: 648dffd3807d4d6c8b02b49f8ad2f6e7
I20260925 02:29:21.083410 25951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:29:21.111073 25951 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:29:21.111635 25951 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:29:21.119788 25951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7745ee55ed41425294f563dd462c70b2: Generated new TSK 0
I20260925 02:29:21.119941 25951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:29:21.904620 25918 catalog_manager.cc:1259] Loaded cluster ID: 648dffd3807d4d6c8b02b49f8ad2f6e7
I20260925 02:29:21.904680 25918 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b: loading cluster ID for follower catalog manager: success
I20260925 02:29:21.905478 25918 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b: acquiring CA information for follower catalog manager: success
I20260925 02:29:21.905756 25918 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 159bc4d0695b4114a33a3f2e55ec505b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:29:21.937557 25933 catalog_manager.cc:1259] Loaded cluster ID: 648dffd3807d4d6c8b02b49f8ad2f6e7
I20260925 02:29:21.937628 25933 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6: loading cluster ID for follower catalog manager: success
I20260925 02:29:21.938372 25933 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6: acquiring CA information for follower catalog manager: success
I20260925 02:29:21.938655 25933 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 75dccfbf1251453bb00393a46bb8dba6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:29:22.173259 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 25672
I20260925 02:29:22.178416 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 25748
I20260925 02:29:22.182861 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 25825
2026-09-25T02:29:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2771 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:29:22 dist-test-slave-dn4n krb5kdc[25967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:29:22 dist-test-slave-dn4n krb5kdc[25967](info): set up 2 sockets
Sep 25 02:29:22 dist-test-slave-dn4n krb5kdc[25967](info): commencing operation
krb5kdc: starting...
W20260925 02:29:23.120806 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:29:23 dist-test-slave-dn4n krb5kdc[25967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:29:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:29:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:29:23.220925 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:44591
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:33251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:44591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:23.291646 25980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:23.291805 25980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:23.291824 25980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:23.292788 25980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:29:23.292817 25980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:23.292830 25980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:23.292841 25980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:29:23.292851 25980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:29:23.294050 25980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:33251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:44591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:44591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:23.294204 25980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:23.294356 25980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:23.298779 25988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:23.298772 25985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:23.298794 25980 server_base.cc:1034] running on GCE node
W20260925 02:29:23.298792 25986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:23.299152 25980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:23.299450 25980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:23.300604 25980 hybrid_clock.cc:648] HybridClock initialized: now 1790303363300588 us; error 32 us; skew 500 ppm
Sep 25 02:29:23 dist-test-slave-dn4n krb5kdc[25967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303363, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:23.302718 25980 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:29:23.303041 25980 webserver.cc:460] Webserver started at http://127.11.220.254:37251/ using document root <none> and password file <none>
I20260925 02:29:23.303264 25980 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:23.303324 25980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:23.303454 25980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:23.304342 25980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "adcf88976562438a9f44cc244870f697"
format_stamp: "Formatted at 2026-09-25 02:29:23 on dist-test-slave-dn4n"
server_key: "938e79bb9a83340992d5191337f4072d"
server_key_iv: "f69f6112fbb210d464e7d5655257cf91"
server_key_version: "encryptionkey@0"
I20260925 02:29:23.304710 25980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "adcf88976562438a9f44cc244870f697"
format_stamp: "Formatted at 2026-09-25 02:29:23 on dist-test-slave-dn4n"
server_key: "938e79bb9a83340992d5191337f4072d"
server_key_iv: "f69f6112fbb210d464e7d5655257cf91"
server_key_version: "encryptionkey@0"
I20260925 02:29:23.305872 25980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:23.306507 25995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:23.306665 25980 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:23.306743 25980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "adcf88976562438a9f44cc244870f697"
format_stamp: "Formatted at 2026-09-25 02:29:23 on dist-test-slave-dn4n"
server_key: "938e79bb9a83340992d5191337f4072d"
server_key_iv: "f69f6112fbb210d464e7d5655257cf91"
server_key_version: "encryptionkey@0"
I20260925 02:29:23.306815 25980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:23.319042 25980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:23.320577 25980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:23.320708 25980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:23.324004 25980 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:44591
I20260925 02:29:23.324033 26049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:44591 every 8 connection(s)
I20260925 02:29:23.324359 25980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:29:23.325158 26050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:23.327142 26050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697: Bootstrap starting.
I20260925 02:29:23.327584 26050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:23.327801 26050 log.cc:824] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:23.328384 26050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697: No bootstrap required, opened a new log
I20260925 02:29:23.329828 26050 raft_consensus.cc:348] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcf88976562438a9f44cc244870f697" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44591 } }
I20260925 02:29:23.329943 26050 raft_consensus.cc:374] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:23.329964 26050 raft_consensus.cc:729] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adcf88976562438a9f44cc244870f697, State: Initialized, Role: FOLLOWER
I20260925 02:29:23.330085 26050 consensus_queue.cc:260] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcf88976562438a9f44cc244870f697" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44591 } }
I20260925 02:29:23.330165 26050 raft_consensus.cc:388] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:29:23.330188 26050 raft_consensus.cc:482] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:29:23.330214 26050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:23.330652 26050 raft_consensus.cc:504] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcf88976562438a9f44cc244870f697" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44591 } }
I20260925 02:29:23.330725 26050 leader_election.cc:302] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adcf88976562438a9f44cc244870f697; no voters:
I20260925 02:29:23.330856 26050 leader_election.cc:288] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:29:23.330885 26053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:29:23.330987 26053 raft_consensus.cc:686] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [term 1 LEADER]: Becoming Leader. State: Replica: adcf88976562438a9f44cc244870f697, State: Running, Role: LEADER
I20260925 02:29:23.331070 26050 sys_catalog.cc:564] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:29:23.331058 26053 consensus_queue.cc:237] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcf88976562438a9f44cc244870f697" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44591 } }
I20260925 02:29:23.331444 26054 sys_catalog.cc:455] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adcf88976562438a9f44cc244870f697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcf88976562438a9f44cc244870f697" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44591 } } }
I20260925 02:29:23.331532 26054 sys_catalog.cc:458] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:23.331604 26055 sys_catalog.cc:455] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adcf88976562438a9f44cc244870f697. Latest consensus state: current_term: 1 leader_uuid: "adcf88976562438a9f44cc244870f697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adcf88976562438a9f44cc244870f697" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44591 } } }
I20260925 02:29:23.331717 26055 sys_catalog.cc:458] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:23.331902 26061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:29:23.332391 26061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:29:23.333782 26061 catalog_manager.cc:1347] Generated new cluster ID: 7b737c8531c84557b6461f7669df6a11
I20260925 02:29:23.333817 26061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:29:23.334427 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 25980
I20260925 02:29:23.334528 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:29:23.334714 12147 external_mini_cluster.cc:1428] Setting key b9a45391b0a91e23b8ff33391dde2d07
I20260925 02:29:23.337852 26061 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:29:23.338584 26061 catalog_manager.cc:1504] Loading token signing keys...
Sep 25 02:29:23 dist-test-slave-dn4n krb5kdc[25967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:23.344060 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:23.335093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33180 (local address 127.11.220.254:44591)
0925 02:29:23.335266 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:23.335274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:23.335344 (+ 70us) server_negotiation.cc:407] Connection header received
0925 02:29:23.335493 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:23.335499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:23.335547 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:23.335674 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:23.336355 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:23.337593 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:23.338494 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:23.338953 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:23.339066 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:23.341286 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:23.341289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:23.341290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:23.341317 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:23.342698 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:23.343022 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:23.343025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:23.343025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:23.343100 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:23.343293 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:23.343294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:23.343295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:23.343466 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:23.343627 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:23.343738 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 02:29:23.343882 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260925 02:29:23.344862 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:29:23.345678 26061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adcf88976562438a9f44cc244870f697: Generated new TSK 0
I20260925 02:29:23.345808 26061 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:29:23.398559 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 25980
2026-09-25T02:29:23Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1216 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:29:23 dist-test-slave-dn4n krb5kdc[26077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:29:23 dist-test-slave-dn4n krb5kdc[26077](info): set up 2 sockets
Sep 25 02:29:23 dist-test-slave-dn4n krb5kdc[26077](info): commencing operation
krb5kdc: starting...
W20260925 02:29:24.336925 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:29:24 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:29:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:29:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:29:24.436775 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:45041
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:45041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:24.508224 26090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:24.508419 26090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:24.508456 26090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:24.509482 26090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:29:24.509550 26090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:24.509583 26090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:24.509621 26090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:29:24.509642 26090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:29:24.510934 26090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:45041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:45041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:24.511173 26090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:24.511373 26090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:24.515710 26095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:24.515815 26090 server_base.cc:1034] running on GCE node
W20260925 02:29:24.515728 26099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:24.515723 26096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:24.516101 26090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:24.516371 26090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:24.517529 26090 hybrid_clock.cc:648] HybridClock initialized: now 1790303364517531 us; error 42 us; skew 500 ppm
Sep 25 02:29:24 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303364, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:24.519816 26090 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:29:24.520191 26090 webserver.cc:460] Webserver started at http://127.11.220.254:37089/ using document root <none> and password file <none>
I20260925 02:29:24.520351 26090 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:24.520387 26090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:24.520511 26090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:24.521356 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "80210798db0aa4b9bc588146f1f2cbde"
server_key_iv: "663194cab71e144347a12ef886528049"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.521600 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "80210798db0aa4b9bc588146f1f2cbde"
server_key_iv: "663194cab71e144347a12ef886528049"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.522707 26090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:29:24.523315 26107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.523485 26090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.523551 26090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "80210798db0aa4b9bc588146f1f2cbde"
server_key_iv: "663194cab71e144347a12ef886528049"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.523655 26090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:24.541620 26090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:24.543390 26090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:24.543475 26090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:24.547036 26090 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:45041
I20260925 02:29:24.547073 26159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:45041 every 8 connection(s)
I20260925 02:29:24.547361 26090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:29:24.548213 26160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:24.550087 26160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Bootstrap starting.
I20260925 02:29:24.550470 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 26090
I20260925 02:29:24.550606 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:29:24.550832 26160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:24.550843 12147 external_mini_cluster.cc:1428] Setting key aa0b2db2f1208e939672ab6cdbd8e1f4
I20260925 02:29:24.551167 26160 log.cc:824] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:24.551837 26160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: No bootstrap required, opened a new log
I20260925 02:29:24.553529 26160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:24.553670 26160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:24.553714 26160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d8f2badcbb4f4ea204aecba8cdcdd4, State: Initialized, Role: FOLLOWER
I20260925 02:29:24.553858 26160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:24.553956 26160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:29:24.553996 26160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:29:24.554039 26160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:24.554492 26160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:24.554600 26160 leader_election.cc:302] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88d8f2badcbb4f4ea204aecba8cdcdd4; no voters:
I20260925 02:29:24.554772 26160 leader_election.cc:288] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:29:24.554803 26165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:29:24.554963 26165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 1 LEADER]: Becoming Leader. State: Replica: 88d8f2badcbb4f4ea204aecba8cdcdd4, State: Running, Role: LEADER
I20260925 02:29:24.554991 26160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:29:24.555058 26165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:24.556401 26166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } } }
I20260925 02:29:24.556583 26166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:24.556586 26165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88d8f2badcbb4f4ea204aecba8cdcdd4. Latest consensus state: current_term: 1 leader_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } } }
Sep 25 02:29:24 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:24.556900 26165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:24.557420 26181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:29:24.557914 26181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:29:24.559237 26181 catalog_manager.cc:1347] Generated new cluster ID: 8390e279021a4f97be0989995c836bc5
I20260925 02:29:24.559279 26181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:29:24.560616 26164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.551304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39146 (local address 127.11.220.254:45041)
0925 02:29:24.551483 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:24.551490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:24.551516 (+ 26us) server_negotiation.cc:407] Connection header received
0925 02:29:24.551893 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:24.551900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:24.551951 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:24.552090 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:24.552686 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.553462 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.554321 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.554770 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.555130 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:24.557526 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:24.557530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:24.557530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:24.557539 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:24.558978 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:24.559336 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:24.559339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:24.559339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:24.559420 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:24.559574 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:24.559574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:24.559574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:24.559788 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:24.559879 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:24.560311 (+ 432us) server_negotiation.cc:299] Negotiation successful
0925 02:29:24.560417 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260925 02:29:24.564788 26181 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:29:24.565392 26181 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:29:24.568943 26181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Generated new TSK 0
I20260925 02:29:24.569096 26181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:29:24.590446 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:45041
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:24.662799 26187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:24.662997 26187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:24.663028 26187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:24.664157 26187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:24.664230 26187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:24.664269 26187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:29:24.665699 26187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:24.665938 26187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:24.666153 26187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:24.670723 26193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:24.670706 26192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:24.670699 26195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:24.670845 26187 server_base.cc:1034] running on GCE node
I20260925 02:29:24.671084 26187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:24.671330 26187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:24.672521 26187 hybrid_clock.cc:648] HybridClock initialized: now 1790303364672506 us; error 29 us; skew 500 ppm
Sep 25 02:29:24 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303364, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:24.674707 26187 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:29:24.675122 26187 webserver.cc:460] Webserver started at http://127.11.220.193:33431/ using document root <none> and password file <none>
I20260925 02:29:24.675331 26187 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:24.675372 26187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:24.675503 26187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:24.676704 26187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "3d843c7dd44b48a3a103e6b98d953fda"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "bbbdfeda9f6b1c9ddc748f0e4c88b264"
server_key_iv: "3222fc11d33bc4433241ed8627ed0557"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.677023 26187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "3d843c7dd44b48a3a103e6b98d953fda"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "bbbdfeda9f6b1c9ddc748f0e4c88b264"
server_key_iv: "3222fc11d33bc4433241ed8627ed0557"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.678251 26187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:29:24.678915 26202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.679091 26187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:24.679178 26187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "3d843c7dd44b48a3a103e6b98d953fda"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "bbbdfeda9f6b1c9ddc748f0e4c88b264"
server_key_iv: "3222fc11d33bc4433241ed8627ed0557"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.679244 26187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:24.706084 26187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:24.707651 26187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:24.707794 26187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:24.708000 26187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:24.708289 26187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:24.708319 26187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.708338 26187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:24.708380 26187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.713788 26187 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:44287
I20260925 02:29:24.713827 26317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:44287 every 8 connection(s)
I20260925 02:29:24.714118 26187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:29:24.714578 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 26187
I20260925 02:29:24.714697 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:29:24.714926 12147 external_mini_cluster.cc:1428] Setting key 9197d4f0b54136b7f65ea52466a2984e
Sep 25 02:29:24 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303364, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
I20260925 02:29:24.723870 26164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.714944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:59445 (local address 127.11.220.254:45041)
0925 02:29:24.714990 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:24.714993 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:24.715216 (+ 223us) server_negotiation.cc:407] Connection header received
0925 02:29:24.715869 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:24.715869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:24.715879 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:24.715913 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:24.717219 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.718012 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.718920 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.719276 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.719350 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:24.721578 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:24.721580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:24.721580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:24.721586 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:24.722815 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:24.723194 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:24.723195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:24.723195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:24.723242 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:24.723382 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:24.723382 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:24.723382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:24.723461 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:24.723500 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:24.723733 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 02:29:24.723811 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:29:24.723991 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.714924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:59445)
0925 02:29:24.715101 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:24.715142 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:24.715732 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:24.715993 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:24.715997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:24.716839 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:24.717118 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:24.717125 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.718103 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.718104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:24.718838 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:24.718840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.719365 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.719366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:24.719456 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:24.719900 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:24.719907 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:24.721491 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:24.722920 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:24.722926 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:24.722927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:24.723126 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:24.723285 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:24.723285 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:24.723285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:24.723353 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:24.723490 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:24.723492 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:24.723579 (+ 87us) client_negotiation.cc:769] Sending connection context
0925 02:29:24.723643 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:29:24.723734 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260925 02:29:24.724526 26318 heartbeater.cc:347] Connected to a master server at 127.11.220.254:45041
I20260925 02:29:24.724619 26318 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:24.724762 26318 heartbeater.cc:511] Master 127.11.220.254:45041 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
I20260925 02:29:24.725308 26124 ts_manager.cc:194] Registered new tserver with Master: 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287)
I20260925 02:29:24.725975 26124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:59445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:29:24.737375 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:45041
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:24.810108 26323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:24.810261 26323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:24.810280 26323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:24.811239 26323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:24.811269 26323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:24.811291 26323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:29:24.812563 26323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:24.812729 26323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:24.812878 26323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:24.817301 26331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:24.817440 26323 server_base.cc:1034] running on GCE node
W20260925 02:29:24.817312 26328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:24.817312 26329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:24.817698 26323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:24.817874 26323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:24.819001 26323 hybrid_clock.cc:648] HybridClock initialized: now 1790303364818976 us; error 42 us; skew 500 ppm
Sep 25 02:29:24 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303364, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:24.821282 26323 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:29:24.821671 26323 webserver.cc:460] Webserver started at http://127.11.220.194:42443/ using document root <none> and password file <none>
I20260925 02:29:24.821872 26323 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:24.821911 26323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:24.822063 26323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:24.822952 26323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "4477257e1a264b8c9a6c0aca0ec74ffd"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "b4c1d2a5dc1cd7fbc7ad3c8f100672ff"
server_key_iv: "147bae7163ba22b9bec845962045597c"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.823271 26323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "4477257e1a264b8c9a6c0aca0ec74ffd"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "b4c1d2a5dc1cd7fbc7ad3c8f100672ff"
server_key_iv: "147bae7163ba22b9bec845962045597c"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.824569 26323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:29:24.825251 26339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.825421 26323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:24.825493 26323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "4477257e1a264b8c9a6c0aca0ec74ffd"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "b4c1d2a5dc1cd7fbc7ad3c8f100672ff"
server_key_iv: "147bae7163ba22b9bec845962045597c"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.825539 26323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:24.834571 26323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:24.836525 26323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:24.836622 26323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:24.836762 26323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:24.837002 26323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:24.837031 26323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.837067 26323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:24.837091 26323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.842139 26323 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:33493
I20260925 02:29:24.842207 26453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:33493 every 8 connection(s)
I20260925 02:29:24.842439 26323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:29:24 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303364, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:24.849637 26164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.843021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:59759 (local address 127.11.220.254:45041)
0925 02:29:24.843065 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:24.843067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:24.843219 (+ 152us) server_negotiation.cc:407] Connection header received
0925 02:29:24.843570 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:24.843571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:24.843580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:24.843625 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:24.844335 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.844855 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.845723 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.845894 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.845939 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:24.847613 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:24.847615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:24.847615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:24.847621 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:24.848734 (+ 1113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:24.849041 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:24.849041 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:24.849042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:24.849083 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:24.849233 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:24.849233 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:24.849233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:24.849316 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:24.849360 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:24.849530 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 02:29:24.849581 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:29:24.849756 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.843001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:59759)
0925 02:29:24.843126 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:24.843153 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:24.843500 (+ 347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:24.843665 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:24.843667 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:24.844120 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:24.844269 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:24.844273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.844945 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.844946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:24.845655 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:24.845656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.845957 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.845958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:24.846054 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:24.846393 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:24.846400 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:24.847540 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:24.848795 (+ 1255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:24.848801 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:24.848803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:24.848986 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:24.849120 (+ 134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:24.849120 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:24.849120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:24.849190 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:24.849358 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:24.849360 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:24.849443 (+ 83us) client_negotiation.cc:769] Sending connection context
0925 02:29:24.849508 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:29:24.849599 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":72,"threads_started":1}
I20260925 02:29:24.850178 26454 heartbeater.cc:347] Connected to a master server at 127.11.220.254:45041
I20260925 02:29:24.850270 26454 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:24.850433 26454 heartbeater.cc:511] Master 127.11.220.254:45041 requested a full tablet report, sending...
I20260925 02:29:24.850755 26124 ts_manager.cc:194] Registered new tserver with Master: 4477257e1a264b8c9a6c0aca0ec74ffd (127.11.220.194:33493)
I20260925 02:29:24.851032 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 26323
I20260925 02:29:24.851106 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:29:24.851099 26124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:59759
I20260925 02:29:24.851281 12147 external_mini_cluster.cc:1428] Setting key 9eebf88ff636fdd1ed8716a53a2c58d5
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:29:24.874253 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:45041
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:24.945245 26459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:24.945441 26459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:24.945477 26459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:24.946554 26459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:24.946610 26459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:24.946661 26459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:29:24.948611 26459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:24.948849 26459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:24.949050 26459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:24.953606 26465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:24.953713 26459 server_base.cc:1034] running on GCE node
W20260925 02:29:24.953603 26467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:24.953599 26464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:24.953976 26459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:24.954209 26459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:24.955338 26459 hybrid_clock.cc:648] HybridClock initialized: now 1790303364955327 us; error 27 us; skew 500 ppm
Sep 25 02:29:24 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303364, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:24.957739 26459 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:29:24.958194 26459 webserver.cc:460] Webserver started at http://127.11.220.195:45425/ using document root <none> and password file <none>
I20260925 02:29:24.958391 26459 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:24.958451 26459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:24.958595 26459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:24.959504 26459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "4592c62e71984bda8343057e444cb3bc"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "075d50cfd0cb34e34e0e45e63932ec29"
server_key_iv: "c710bf6a2efb5a26b53522565a01ca60"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.959847 26459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "4592c62e71984bda8343057e444cb3bc"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "075d50cfd0cb34e34e0e45e63932ec29"
server_key_iv: "c710bf6a2efb5a26b53522565a01ca60"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.961163 26459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:29:24.961771 26475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.961915 26459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:29:24.961997 26459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "4592c62e71984bda8343057e444cb3bc"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "075d50cfd0cb34e34e0e45e63932ec29"
server_key_iv: "c710bf6a2efb5a26b53522565a01ca60"
server_key_version: "encryptionkey@0"
I20260925 02:29:24.962069 26459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:24.969924 26459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:24.971426 26459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:24.971560 26459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:24.971745 26459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:24.972034 26459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:24.972082 26459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.972117 26459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:24.972150 26459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:24.977355 26459 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:40461
I20260925 02:29:24.977385 26589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:40461 every 8 connection(s)
I20260925 02:29:24.977689 26459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:29:24 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303364, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:24.985278 26164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.978339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:57387 (local address 127.11.220.254:45041)
0925 02:29:24.978388 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:24.978391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:24.978584 (+ 193us) server_negotiation.cc:407] Connection header received
0925 02:29:24.978990 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:24.978991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:24.979000 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:24.979049 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:24.979822 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.980303 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.981225 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.981394 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.981441 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:24.983241 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:24.983243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:24.983243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:24.983248 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:24.984281 (+ 1033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:24.984653 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:24.984654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:24.984654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:24.984697 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:24.984870 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:24.984870 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:24.984871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:24.984948 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:24.984988 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:24.985171 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 02:29:24.985221 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:29:24.985459 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.978328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:57387)
0925 02:29:24.978481 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:24.978511 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:24.978919 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:24.979097 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:24.979100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:24.979596 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:24.979756 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:24.979760 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.980374 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.980374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:24.981160 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:24.981162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.981458 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.981459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:24.981539 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:24.981997 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:24.982004 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:24.983167 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:24.984341 (+ 1174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:24.984347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:24.984348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:24.984596 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:24.984752 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:24.984752 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:24.984753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:24.984835 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:24.984993 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:24.984997 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:24.985092 (+ 95us) client_negotiation.cc:769] Sending connection context
0925 02:29:24.985159 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 02:29:24.985252 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260925 02:29:24.985896 26590 heartbeater.cc:347] Connected to a master server at 127.11.220.254:45041
I20260925 02:29:24.985975 26590 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:24.986086 26590 heartbeater.cc:511] Master 127.11.220.254:45041 requested a full tablet report, sending...
I20260925 02:29:24.986390 26124 ts_manager.cc:194] Registered new tserver with Master: 4592c62e71984bda8343057e444cb3bc (127.11.220.195:40461)
I20260925 02:29:24.986724 26124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:57387
I20260925 02:29:24.987769 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 26459
I20260925 02:29:24.987843 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:29:24.987953 12147 external_mini_cluster.cc:1428] Setting key 2d777ae5fae11ec964246fcc1318c603
I20260925 02:29:24.988724 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:29:24.994341 26164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.989924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39162 (local address 127.11.220.254:45041)
0925 02:29:24.989966 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:24.989968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:24.990044 (+ 76us) server_negotiation.cc:407] Connection header received
0925 02:29:24.990093 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:24.990094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:24.990101 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:24.990149 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:24.990590 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.990970 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.991568 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:24.991718 (+ 150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:24.991759 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:24.992680 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:24.992681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:24.992681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:24.992686 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:24.993572 (+ 886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:24.993842 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:24.993842 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:24.993842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:24.993883 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:24.993983 (+ 100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:24.993984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:24.993984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:24.994080 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:24.994147 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:24.994214 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 02:29:24.994279 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 02:29:24.995733 26124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:29:24.997314 26124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:29:25.002931 26599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.999233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44287 (local address 127.0.0.1:52152)
0925 02:29:24.999497 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:24.999506 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:24.999553 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:24.999890 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:24.999890 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:24.999892 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:24.999947 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:24.999948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.000993 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.000993 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.002094 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.002097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.002682 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.002683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.002755 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.002766 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.002827 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.002871 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":28,"thread_start_us":71,"threads_started":1}
I20260925 02:29:25.002942 26601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.999327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.11.220.193:44287)
0925 02:29:24.999749 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:24.999753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:24.999761 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:29:24.999793 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:24.999795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:24.999807 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:24.999884 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:25.000059 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.000877 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.002182 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.002604 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.002685 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.002734 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.002840 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.002874 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":77,"threads_started":1}
I20260925 02:29:25.003049 26602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.999406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:33493 (local address 127.0.0.1:33658)
0925 02:29:24.999994 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.000003 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.000067 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.000263 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.000263 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.000265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:25.000296 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.000297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.001292 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.001293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.001976 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.001978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.002819 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.002820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.002875 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.002884 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.002925 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.002965 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":5,"thread_start_us":54,"threads_started":1}
I20260925 02:29:25.003255 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.999265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35452 (local address 127.11.220.195:40461)
0925 02:29:24.999569 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:24.999574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:24.999595 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:29:24.999625 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:24.999627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:24.999640 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:24.999711 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:24.999848 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.000856 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.002163 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.002716 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.002765 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.002802 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.003158 (+ 356us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.003194 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":61,"threads_started":1}
I20260925 02:29:25.003263 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.999518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33658 (local address 127.11.220.194:33493)
0925 02:29:24.999678 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:24.999699 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.000120 (+ 421us) server_negotiation.cc:407] Connection header received
0925 02:29:25.000176 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.000178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.000191 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.000250 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:25.000343 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.001199 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.002050 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.002670 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.002736 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.002785 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.003159 (+ 374us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.003198 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260925 02:29:25.003439 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:24.999221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40461 (local address 127.0.0.1:35452)
0925 02:29:24.999363 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:24.999394 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:24.999454 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:24.999727 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:24.999730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:24.999732 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:24.999782 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:24.999783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.000953 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.000955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.002101 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.002103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.002826 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.002827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.002875 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.002883 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.002925 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.002965 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":57,"thread_start_us":53,"threads_started":1}
I20260925 02:29:25.003690 26250 tablet_service.cc:1467] Processing CreateTablet for tablet 40864cf94bb74722a58eacbc5cf1c51a (DEFAULT_TABLE table=test-table [id=f9d79818f4094bc9a8a93d0db17b16aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:25.003993 26250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40864cf94bb74722a58eacbc5cf1c51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:25.003921 26388 tablet_service.cc:1467] Processing CreateTablet for tablet 40864cf94bb74722a58eacbc5cf1c51a (DEFAULT_TABLE table=test-table [id=f9d79818f4094bc9a8a93d0db17b16aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:25.003971 26524 tablet_service.cc:1467] Processing CreateTablet for tablet 40864cf94bb74722a58eacbc5cf1c51a (DEFAULT_TABLE table=test-table [id=f9d79818f4094bc9a8a93d0db17b16aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:25.004191 26388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40864cf94bb74722a58eacbc5cf1c51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:25.004212 26524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40864cf94bb74722a58eacbc5cf1c51a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:25.006604 26604 tablet_bootstrap.cc:492] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: Bootstrap starting.
I20260925 02:29:25.007035 26605 tablet_bootstrap.cc:492] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Bootstrap starting.
I20260925 02:29:25.007403 26606 tablet_bootstrap.cc:492] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: Bootstrap starting.
I20260925 02:29:25.007560 26604 tablet_bootstrap.cc:654] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:25.007565 26605 tablet_bootstrap.cc:654] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:25.007977 26606 tablet_bootstrap.cc:654] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:25.008185 26604 log.cc:824] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:25.008225 26605 log.cc:824] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:25.008591 26606 log.cc:824] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:25.008981 26605 tablet_bootstrap.cc:492] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: No bootstrap required, opened a new log
I20260925 02:29:25.008991 26604 tablet_bootstrap.cc:492] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: No bootstrap required, opened a new log
I20260925 02:29:25.009052 26604 ts_tablet_manager.cc:1397] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:29:25.009052 26605 ts_tablet_manager.cc:1397] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:29:25.009224 26606 tablet_bootstrap.cc:492] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: No bootstrap required, opened a new log
I20260925 02:29:25.009294 26606 ts_tablet_manager.cc:1397] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:29:25.010541 26606 raft_consensus.cc:348] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.010699 26606 raft_consensus.cc:374] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:25.010737 26606 raft_consensus.cc:729] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4477257e1a264b8c9a6c0aca0ec74ffd, State: Initialized, Role: FOLLOWER
I20260925 02:29:25.010807 26605 raft_consensus.cc:348] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.010870 26606 consensus_queue.cc:260] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.010866 26604 raft_consensus.cc:348] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.010955 26605 raft_consensus.cc:374] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:25.010982 26604 raft_consensus.cc:374] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:25.010989 26605 raft_consensus.cc:729] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d843c7dd44b48a3a103e6b98d953fda, State: Initialized, Role: FOLLOWER
I20260925 02:29:25.011020 26604 raft_consensus.cc:729] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4592c62e71984bda8343057e444cb3bc, State: Initialized, Role: FOLLOWER
I20260925 02:29:25.011142 26606 ts_tablet_manager.cc:1428] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:29:25.011106 26605 consensus_queue.cc:260] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.011119 26604 consensus_queue.cc:260] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.011286 26454 heartbeater.cc:503] Master 127.11.220.254:45041 was elected leader, sending a full tablet report...
I20260925 02:29:25.011404 26604 ts_tablet_manager.cc:1428] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:29:25.011435 26590 heartbeater.cc:503] Master 127.11.220.254:45041 was elected leader, sending a full tablet report...
I20260925 02:29:25.011775 26605 ts_tablet_manager.cc:1428] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:29:25.011825 26318 heartbeater.cc:503] Master 127.11.220.254:45041 was elected leader, sending a full tablet report...
W20260925 02:29:25.092912 26455 tablet.cc:2369] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:29:25.214794 26319 tablet.cc:2369] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:29:25.228235 26591 tablet.cc:2369] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:29:25.231873 26611 raft_consensus.cc:482] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:29:25.231985 26611 raft_consensus.cc:504] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.232357 26611 leader_election.cc:288] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287), 4477257e1a264b8c9a6c0aca0ec74ffd (127.11.220.194:33493)
I20260925 02:29:25.235985 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.232787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:46281 (local address 127.11.220.194:33493)
0925 02:29:25.232858 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.232861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.232957 (+ 96us) server_negotiation.cc:407] Connection header received
0925 02:29:25.233029 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.233030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.233041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.233106 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:25.233231 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.233888 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.235124 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.235667 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.235737 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.235744 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.235886 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.235923 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:29:25.235985 26601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.232604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:44761 (local address 127.11.220.193:44287)
0925 02:29:25.232669 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.232671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.232679 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:29:25.232754 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.232755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.232765 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.232849 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:25.233051 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.233801 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.235129 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.235667 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.235737 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.235744 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.235887 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.235923 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:29:25.236009 26592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.232575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44287 (local address 127.11.220.195:44761)
0925 02:29:25.232640 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.232650 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.232693 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.232894 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.232895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.232898 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:25.232975 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.232976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.233878 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.233878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.235037 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.235040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.235743 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.235744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.235818 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.235820 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.235868 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.235924 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":15}
I20260925 02:29:25.236016 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.232712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:33493 (local address 127.11.220.195:46281)
0925 02:29:25.232891 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.232899 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.232954 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.233118 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.233118 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.233119 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:25.233173 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.233174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.233945 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.233946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.235038 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.235040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.235743 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.235744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.235812 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.235816 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.235868 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.235924 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":16,"thread_start_us":110,"threads_started":1}
I20260925 02:29:25.236382 26408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40864cf94bb74722a58eacbc5cf1c51a" candidate_uuid: "4592c62e71984bda8343057e444cb3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" is_pre_election: true
I20260925 02:29:25.236382 26272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40864cf94bb74722a58eacbc5cf1c51a" candidate_uuid: "4592c62e71984bda8343057e444cb3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d843c7dd44b48a3a103e6b98d953fda" is_pre_election: true
I20260925 02:29:25.236548 26272 raft_consensus.cc:2454] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4592c62e71984bda8343057e444cb3bc in term 0.
I20260925 02:29:25.236548 26408 raft_consensus.cc:2454] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4592c62e71984bda8343057e444cb3bc in term 0.
I20260925 02:29:25.236771 26478 leader_election.cc:302] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4477257e1a264b8c9a6c0aca0ec74ffd, 4592c62e71984bda8343057e444cb3bc; no voters:
I20260925 02:29:25.236886 26611 raft_consensus.cc:2781] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:29:25.236939 26611 raft_consensus.cc:482] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:29:25.236961 26611 raft_consensus.cc:3037] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:25.237582 26611 raft_consensus.cc:504] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.237697 26611 leader_election.cc:288] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [CANDIDATE]: Term 1 election: Requested vote from peers 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287), 4477257e1a264b8c9a6c0aca0ec74ffd (127.11.220.194:33493)
I20260925 02:29:25.237867 26408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40864cf94bb74722a58eacbc5cf1c51a" candidate_uuid: "4592c62e71984bda8343057e444cb3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd"
I20260925 02:29:25.237854 26272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40864cf94bb74722a58eacbc5cf1c51a" candidate_uuid: "4592c62e71984bda8343057e444cb3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d843c7dd44b48a3a103e6b98d953fda"
I20260925 02:29:25.237932 26272 raft_consensus.cc:3037] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:25.237932 26408 raft_consensus.cc:3037] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:25.238751 26272 raft_consensus.cc:2454] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4592c62e71984bda8343057e444cb3bc in term 1.
I20260925 02:29:25.238751 26408 raft_consensus.cc:2454] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4592c62e71984bda8343057e444cb3bc in term 1.
I20260925 02:29:25.238911 26478 leader_election.cc:302] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4477257e1a264b8c9a6c0aca0ec74ffd, 4592c62e71984bda8343057e444cb3bc; no voters:
I20260925 02:29:25.238994 26611 raft_consensus.cc:2781] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:29:25.239114 26611 raft_consensus.cc:686] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 1 LEADER]: Becoming Leader. State: Replica: 4592c62e71984bda8343057e444cb3bc, State: Running, Role: LEADER
I20260925 02:29:25.239207 26611 consensus_queue.cc:237] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.239912 26124 catalog_manager.cc:5495] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 4592c62e71984bda8343057e444cb3bc (127.11.220.195). New cstate: current_term: 1 leader_uuid: "4592c62e71984bda8343057e444cb3bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } health_report { overall_health: UNKNOWN } } }
I20260925 02:29:25.284317 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.281964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.11.220.195:40461)
0925 02:29:25.282031 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.282033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.282042 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:29:25.282117 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.282118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.282156 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.282223 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:29:25.282350 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.282855 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.283552 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.283733 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.283803 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.283958 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:29:25.284059 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:29:25.284175 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.284193 (+ 18us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.284212 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:29:25.286130 26408 raft_consensus.cc:1261] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Refusing update from remote peer 4592c62e71984bda8343057e444cb3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:29:25.286154 26272 raft_consensus.cc:1261] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Refusing update from remote peer 4592c62e71984bda8343057e444cb3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:29:25.286476 26614 consensus_queue.cc:1035] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:29:25.286612 26611 consensus_queue.cc:1035] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:29:25.288601 26617 mvcc.cc:205] Tried to move back new op lower bound from 7333082584209096704 to 7333082584020242432. Current Snapshot: MvccSnapshot[applied={T|T < 7333082584209096704}]
I20260925 02:29:25.288901 26620 mvcc.cc:205] Tried to move back new op lower bound from 7333082584209096704 to 7333082584020242432. Current Snapshot: MvccSnapshot[applied={T|T < 7333082584209096704}]
I20260925 02:29:25.289309 26619 mvcc.cc:205] Tried to move back new op lower bound from 7333082584209096704 to 7333082584020242432. Current Snapshot: MvccSnapshot[applied={T|T < 7333082584209096704}]
I20260925 02:29:25.291108 26124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:29:25.291199 26124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:29:25.291906 26124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Sending DeleteTablet for 3 replicas of tablet 40864cf94bb74722a58eacbc5cf1c51a
I20260925 02:29:25.292233 26524 tablet_service.cc:1514] Processing DeleteTablet for tablet 40864cf94bb74722a58eacbc5cf1c51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:25 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:35452
I20260925 02:29:25.292238 26250 tablet_service.cc:1514] Processing DeleteTablet for tablet 40864cf94bb74722a58eacbc5cf1c51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:25 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:52152
I20260925 02:29:25.292274 26388 tablet_service.cc:1514] Processing DeleteTablet for tablet 40864cf94bb74722a58eacbc5cf1c51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:25 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:33658
I20260925 02:29:25.292469 26629 tablet_replica.cc:330] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: stopping tablet replica
I20260925 02:29:25.292484 26628 tablet_replica.cc:330] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: stopping tablet replica
I20260925 02:29:25.292557 26629 raft_consensus.cc:2229] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:29:25.292557 26628 raft_consensus.cc:2229] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:29:25.292539 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 26187
I20260925 02:29:25.293314 26630 tablet_replica.cc:330] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: stopping tablet replica
I20260925 02:29:25.293390 26630 raft_consensus.cc:2229] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:29:25.293462 26630 raft_consensus.cc:2258] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:29:25.293488 26629 raft_consensus.cc:2258] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:29:25.293882 26630 ts_tablet_manager.cc:1905] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:29:25.293973 26629 ts_tablet_manager.cc:1905] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:29:25.295665 26629 ts_tablet_manager.cc:1918] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:29:25.295729 26630 ts_tablet_manager.cc:1918] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:29:25.295748 26629 log.cc:1196] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/40864cf94bb74722a58eacbc5cf1c51a
I20260925 02:29:25.295782 26630 log.cc:1196] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/40864cf94bb74722a58eacbc5cf1c51a
I20260925 02:29:25.296020 26629 ts_tablet_manager.cc:1939] T 40864cf94bb74722a58eacbc5cf1c51a P 4592c62e71984bda8343057e444cb3bc: Deleting consensus metadata
I20260925 02:29:25.296020 26630 ts_tablet_manager.cc:1939] T 40864cf94bb74722a58eacbc5cf1c51a P 4477257e1a264b8c9a6c0aca0ec74ffd: Deleting consensus metadata
I20260925 02:29:25.296350 26111 catalog_manager.cc:4841] TS 4592c62e71984bda8343057e444cb3bc (127.11.220.195:40461): tablet 40864cf94bb74722a58eacbc5cf1c51a (table test-table [id=f9d79818f4094bc9a8a93d0db17b16aa]) successfully deleted
I20260925 02:29:25.296350 26110 catalog_manager.cc:4841] TS 4477257e1a264b8c9a6c0aca0ec74ffd (127.11.220.194:33493): tablet 40864cf94bb74722a58eacbc5cf1c51a (table test-table [id=f9d79818f4094bc9a8a93d0db17b16aa]) successfully deleted
W20260925 02:29:25.299744 26110 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:44287 (error 108)
I20260925 02:29:25.299944 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 26323
I20260925 02:29:25.300691 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.300101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44287 (local address 127.0.0.1:52154)
0925 02:29:25.300633 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.300666 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:44287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515}
W20260925 02:29:25.300946 26110 catalog_manager.cc:4584] TS 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40864cf94bb74722a58eacbc5cf1c51a: Network error: Client connection negotiation failed: client connection to 127.11.220.193:44287: connect: Connection refused (error 111)
I20260925 02:29:25.304751 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 26459
I20260925 02:29:25.309463 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 26090
I20260925 02:29:25.314343 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:45041
--webserver_interface=127.11.220.254
--webserver_port=37089
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:45041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:25.386750 26632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:25.386976 26632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:25.387012 26632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:25.388060 26632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:29:25.388134 26632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:25.388159 26632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:25.388188 26632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:29:25.388207 26632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:29:25.389604 26632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:45041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:45041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=37089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:25.389851 26632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:25.390044 26632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:25.398636 26641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:25.398670 26638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:25.398636 26639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:25.398881 26632 server_base.cc:1034] running on GCE node
I20260925 02:29:25.399096 26632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:25.399377 26632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:25.400518 26632 hybrid_clock.cc:648] HybridClock initialized: now 1790303365400505 us; error 23 us; skew 500 ppm
Sep 25 02:29:25 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:25.402653 26632 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:29:25.403052 26632 webserver.cc:460] Webserver started at http://127.11.220.254:37089/ using document root <none> and password file <none>
I20260925 02:29:25.403285 26632 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:25.403355 26632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:25.404636 26632 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:25.405262 26648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.405424 26632 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:25.405507 26632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "80210798db0aa4b9bc588146f1f2cbde"
server_key_iv: "663194cab71e144347a12ef886528049"
server_key_version: "encryptionkey@0"
I20260925 02:29:25.405902 26632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:25.423770 26632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:25.425400 26632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:25.425530 26632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:25.429139 26632 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:45041
I20260925 02:29:25.429214 26701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:45041 every 8 connection(s)
I20260925 02:29:25.429569 26632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:29:25.430974 26702 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:29:25.431481 26702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Bootstrap starting.
I20260925 02:29:25.438706 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 26632
I20260925 02:29:25.438930 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:44287
--local_ip_for_outbound_sockets=127.11.220.193
--tserver_master_addrs=127.11.220.254:45041
--webserver_port=33431
--webserver_interface=127.11.220.193
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:29:25.439425 26702 log.cc:824] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:25.441499 26702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:29:25.441773 26702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Bootstrap complete.
I20260925 02:29:25.443682 26702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:25.443887 26702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d8f2badcbb4f4ea204aecba8cdcdd4, State: Initialized, Role: FOLLOWER
I20260925 02:29:25.444056 26702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:25.444135 26702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:29:25.444185 26702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:29:25.444226 26702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:29:25.445212 26702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:25.445322 26702 leader_election.cc:302] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88d8f2badcbb4f4ea204aecba8cdcdd4; no voters:
I20260925 02:29:25.445465 26702 leader_election.cc:288] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 02:29:25.445497 26706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 02:29:25.445597 26706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 2 LEADER]: Becoming Leader. State: Replica: 88d8f2badcbb4f4ea204aecba8cdcdd4, State: Running, Role: LEADER
I20260925 02:29:25.445725 26702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:29:25.445750 26706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:25.446076 26708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } } }
I20260925 02:29:25.446157 26708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:25.446373 26707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88d8f2badcbb4f4ea204aecba8cdcdd4. Latest consensus state: current_term: 2 leader_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } } }
I20260925 02:29:25.446448 26707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:25.447259 26722 catalog_manager.cc:1259] Loaded cluster ID: 8390e279021a4f97be0989995c836bc5
I20260925 02:29:25.447295 26722 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: loading cluster ID for follower catalog manager: success
W20260925 02:29:25.447471 26722 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260925 02:29:25.447500 26722 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260925 02:29:25.447542 26714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:29:25.447901 26714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:29:25.447970 26714 catalog_manager.cc:1259] Loaded cluster ID: 8390e279021a4f97be0989995c836bc5
I20260925 02:29:25.447997 26714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:29:25.448170 26714 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 02:29:25.452567 26714 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 02:29:25.454766 26714 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:29:25.454936 26714 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Loaded TSK: 0
I20260925 02:29:25.455147 26714 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:29:25.514478 26705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:25.514673 26705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:25.514708 26705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:25.515726 26705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:25.515780 26705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:25.515817 26705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:29:25.517194 26705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:44287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=33431
--tserver_master_addrs=127.11.220.254:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:25.517426 26705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:25.517633 26705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:25.522190 26730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:25.522162 26733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:25.522277 26731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:25.522553 26705 server_base.cc:1034] running on GCE node
I20260925 02:29:25.522743 26705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:25.522985 26705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:25.524151 26705 hybrid_clock.cc:648] HybridClock initialized: now 1790303365524135 us; error 35 us; skew 500 ppm
Sep 25 02:29:25 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:25.526750 26705 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:29:25.527122 26705 webserver.cc:460] Webserver started at http://127.11.220.193:33431/ using document root <none> and password file <none>
I20260925 02:29:25.527333 26705 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:25.527416 26705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:25.528594 26705 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:29:25.529215 26740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.529390 26705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.529475 26705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "3d843c7dd44b48a3a103e6b98d953fda"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "bbbdfeda9f6b1c9ddc748f0e4c88b264"
server_key_iv: "3222fc11d33bc4433241ed8627ed0557"
server_key_version: "encryptionkey@0"
I20260925 02:29:25.529776 26705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:25.549012 26705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:25.551211 26705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:25.551411 26705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:25.551632 26705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:25.552043 26747 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 02:29:25.553925 26705 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 02:29:25.553978 26705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 02:29:25.554018 26705 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 02:29:25.554615 26705 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 02:29:25.554663 26705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 02:29:25.554739 26747 tablet_bootstrap.cc:492] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Bootstrap starting.
I20260925 02:29:25.561093 26705 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:44287
I20260925 02:29:25.561415 26705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:29:25.563665 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 26705
I20260925 02:29:25.563915 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:33493
--local_ip_for_outbound_sockets=127.11.220.194
--tserver_master_addrs=127.11.220.254:45041
--webserver_port=42443
--webserver_interface=127.11.220.194
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:29:25.564404 26747 log.cc:824] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:25.566475 26857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:44287 every 8 connection(s)
I20260925 02:29:25.567723 26747 tablet_bootstrap.cc:492] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:29:25.567992 26747 tablet_bootstrap.cc:492] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Bootstrap complete.
I20260925 02:29:25.569198 26747 ts_tablet_manager.cc:1397] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.005s
I20260925 02:29:25.570760 26747 raft_consensus.cc:348] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.571085 26747 raft_consensus.cc:729] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d843c7dd44b48a3a103e6b98d953fda, State: Initialized, Role: FOLLOWER
I20260925 02:29:25.571251 26747 consensus_queue.cc:260] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } } peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } }
I20260925 02:29:25.571547 26747 ts_tablet_manager.cc:1428] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Sep 25 02:29:25 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:25.579095 26861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.562081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:40891 (local address 127.11.220.254:45041)
0925 02:29:25.562295 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.562314 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.565511 (+ 3197us) server_negotiation.cc:407] Connection header received
0925 02:29:25.566501 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.566508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.566565 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.566725 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:25.567870 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.569254 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.570768 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.571360 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.571468 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.575331 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:25.575335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:25.575335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:25.575346 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:25.577476 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:25.577841 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:25.577861 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:25.577862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:25.577994 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:25.578245 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:25.578246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:25.578247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:25.578464 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:25.578547 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.578819 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.578929 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260925 02:29:25.579095 26860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.562034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:40891)
0925 02:29:25.564961 (+ 2927us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.564999 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.566249 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.566998 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.567002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.567424 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:25.567727 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.567733 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.569379 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.569381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.570667 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.570671 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.572543 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.572544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.572681 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.573414 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:25.573424 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:25.575202 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:25.577564 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:25.577570 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:25.577572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:25.577761 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:25.578068 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:25.578069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:25.578069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:25.578171 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:25.578539 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:25.578545 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:25.578670 (+ 125us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.578766 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.578918 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2894,"thread_start_us":94,"threads_started":1}
I20260925 02:29:25.580078 26858 heartbeater.cc:347] Connected to a master server at 127.11.220.254:45041
I20260925 02:29:25.580262 26858 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:25.580518 26858 heartbeater.cc:511] Master 127.11.220.254:45041 requested a full tablet report, sending...
I20260925 02:29:25.581312 26666 ts_manager.cc:194] Registered new tserver with Master: 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287)
I20260925 02:29:25.582415 26666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:40891
I20260925 02:29:25.583283 26858 heartbeater.cc:503] Master 127.11.220.254:45041 was elected leader, sending a full tablet report...
Sep 25 02:29:25 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:29:25.589439 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.581907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44287 (local address 127.0.0.1:52160)
0925 02:29:25.582166 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.582190 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.582277 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.582450 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.582454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.582848 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:25.582899 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.582902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.583976 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.583977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.584586 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.584588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.585050 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.585051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.585138 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.585553 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:25.585557 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:25.586769 (+ 1212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:25.588355 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:25.588362 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:25.588364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:25.588618 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:25.588826 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:25.588827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:25.588827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:25.588900 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:25.589219 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:25.589224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:25.589260 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.589314 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.589385 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":80,"threads_started":1}
I20260925 02:29:25.589509 26866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.581956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.11.220.193:44287)
0925 02:29:25.582114 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.582117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.582259 (+ 142us) server_negotiation.cc:407] Connection header received
0925 02:29:25.582368 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.582371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.582383 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.582445 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:25.582999 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.583902 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.584674 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.584952 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.585045 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.586873 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:25.586880 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:25.586881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:25.586887 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:25.588252 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:25.588695 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:25.588698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:25.588698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:25.588756 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:25.588973 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:25.588975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:25.588975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:25.589156 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:25.589232 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.589359 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.589438 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260925 02:29:25.589865 26771 tablet_service.cc:1514] Processing DeleteTablet for tablet 40864cf94bb74722a58eacbc5cf1c51a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:25 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:52160
I20260925 02:29:25.590329 26867 tablet_replica.cc:330] stopping tablet replica
I20260925 02:29:25.590404 26867 raft_consensus.cc:2229] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:29:25.590474 26867 raft_consensus.cc:2258] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:29:25.590739 26867 ts_tablet_manager.cc:1905] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:29:25.592787 26867 ts_tablet_manager.cc:1918] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:29:25.592840 26867 log.cc:1196] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/40864cf94bb74722a58eacbc5cf1c51a
I20260925 02:29:25.593040 26867 ts_tablet_manager.cc:1939] T 40864cf94bb74722a58eacbc5cf1c51a P 3d843c7dd44b48a3a103e6b98d953fda: Deleting consensus metadata
I20260925 02:29:25.593439 26651 catalog_manager.cc:4841] TS 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287): tablet 40864cf94bb74722a58eacbc5cf1c51a (table test-table [id=f9d79818f4094bc9a8a93d0db17b16aa]) successfully deleted
W20260925 02:29:25.647240 26863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:25.647419 26863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:25.647450 26863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:25.648423 26863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:25.648497 26863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:25.648545 26863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:29:25.649886 26863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:33493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=42443
--tserver_master_addrs=127.11.220.254:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:25.650130 26863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:25.650338 26863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:25.654663 26875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:25.654732 26873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:25.654675 26872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:25.654888 26863 server_base.cc:1034] running on GCE node
I20260925 02:29:25.655122 26863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:25.655345 26863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:25.656502 26863 hybrid_clock.cc:648] HybridClock initialized: now 1790303365656484 us; error 44 us; skew 500 ppm
Sep 25 02:29:25 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:25.658627 26863 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:29:25.658948 26863 webserver.cc:460] Webserver started at http://127.11.220.194:42443/ using document root <none> and password file <none>
I20260925 02:29:25.659157 26863 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:25.659206 26863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:25.660352 26863 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:25.660928 26883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.661087 26863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.661172 26863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "4477257e1a264b8c9a6c0aca0ec74ffd"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "b4c1d2a5dc1cd7fbc7ad3c8f100672ff"
server_key_iv: "147bae7163ba22b9bec845962045597c"
server_key_version: "encryptionkey@0"
I20260925 02:29:25.661489 26863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:25.680927 26863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:25.682412 26863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:25.682502 26863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:25.682648 26863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:25.682855 26863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:25.682880 26863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.682897 26863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:25.682909 26863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.688061 26863 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:33493
I20260925 02:29:25.688143 26997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:33493 every 8 connection(s)
I20260925 02:29:25.688437 26863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:29:25.691908 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 26863
I20260925 02:29:25.692154 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:40461
--local_ip_for_outbound_sockets=127.11.220.195
--tserver_master_addrs=127.11.220.254:45041
--webserver_port=45425
--webserver_interface=127.11.220.195
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 02:29:25 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:25.697611 27000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.689182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:35009)
0925 02:29:25.689310 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.689340 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.689763 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.689984 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.689988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.690545 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:25.690701 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.690705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.691333 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.691334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.692101 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.692103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.692623 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.692624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.692749 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.693197 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:25.693205 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:25.694625 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:25.696372 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:25.696378 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:25.696380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:25.696644 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:25.696845 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:25.696845 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:25.696845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:25.696945 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:25.697218 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:25.697222 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:25.697314 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.697383 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.697480 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260925 02:29:25.697656 26861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.689167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:35009 (local address 127.11.220.254:45041)
0925 02:29:25.689205 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.689207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.689412 (+ 205us) server_negotiation.cc:407] Connection header received
0925 02:29:25.689859 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.689861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.689876 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.689927 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:25.690792 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.691245 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.692191 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.692529 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.692602 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.694717 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:25.694719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:25.694719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:25.694726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:25.696295 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:25.696710 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:25.696712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:25.696713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:25.696775 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:25.697011 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:25.697012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:25.697012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:25.697151 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:25.697225 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.697427 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.697521 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:29:25.698033 26998 heartbeater.cc:347] Connected to a master server at 127.11.220.254:45041
I20260925 02:29:25.698129 26998 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:25.698309 26998 heartbeater.cc:511] Master 127.11.220.254:45041 requested a full tablet report, sending...
I20260925 02:29:25.698696 26666 ts_manager.cc:194] Registered new tserver with Master: 4477257e1a264b8c9a6c0aca0ec74ffd (127.11.220.194:33493)
I20260925 02:29:25.699069 26666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:35009
W20260925 02:29:25.766921 27001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:25.767071 27001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:25.767091 27001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:25.768040 27001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:25.768074 27001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:25.768095 27001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:29:25.769498 27001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:40461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=45425
--tserver_master_addrs=127.11.220.254:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:25.769702 27001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:25.769919 27001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:25.774320 27006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:25.774437 27001 server_base.cc:1034] running on GCE node
W20260925 02:29:25.774296 27007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:25.774296 27009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:25.774724 27001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:25.774961 27001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:25.776124 27001 hybrid_clock.cc:648] HybridClock initialized: now 1790303365776119 us; error 33 us; skew 500 ppm
Sep 25 02:29:25 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:25.783277 27001 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:29:25.783650 27001 webserver.cc:460] Webserver started at http://127.11.220.195:45425/ using document root <none> and password file <none>
I20260925 02:29:25.783855 27001 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:25.783923 27001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:25.785154 27001 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260925 02:29:25.785691 27016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.785856 27001 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:29:25.785944 27001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "4592c62e71984bda8343057e444cb3bc"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "075d50cfd0cb34e34e0e45e63932ec29"
server_key_iv: "c710bf6a2efb5a26b53522565a01ca60"
server_key_version: "encryptionkey@0"
I20260925 02:29:25.786208 27001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:25.799526 27001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:25.801074 27001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:25.801209 27001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:25.801398 27001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:25.801666 27001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:25.801718 27001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.801753 27001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:25.801779 27001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.807029 27001 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:40461
I20260925 02:29:25.807068 27131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:40461 every 8 connection(s)
I20260925 02:29:25.807368 27001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:29:25 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:25.814812 26861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.808000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:39727 (local address 127.11.220.254:45041)
0925 02:29:25.808055 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.808057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.808228 (+ 171us) server_negotiation.cc:407] Connection header received
0925 02:29:25.808662 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.808663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.808672 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.808718 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:25.809548 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.810008 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.810869 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.811044 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.811088 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.812782 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:25.812783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:25.812784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:25.812790 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:25.813824 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:25.814132 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:25.814132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:25.814132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:25.814176 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:25.814326 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:25.814327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:25.814327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:25.814412 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:25.814456 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.814694 (+ 238us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.814754 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 02:29:25.814930 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.807981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:39727)
0925 02:29:25.808116 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.808150 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.808582 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.808758 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.808761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.809297 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:25.809451 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.809455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.810082 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.810082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.810798 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.810800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.811107 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.811108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.811189 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.811529 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:25.811536 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:25.812707 (+ 1171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:25.813884 (+ 1177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:25.813890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:25.813891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:25.814073 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:25.814213 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:25.814213 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:25.814213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:25.814284 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:25.814462 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:25.814467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:25.814594 (+ 127us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.814668 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.814767 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":78,"threads_started":1}
I20260925 02:29:25.815424 27132 heartbeater.cc:347] Connected to a master server at 127.11.220.254:45041
I20260925 02:29:25.815496 27132 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:25.815665 27132 heartbeater.cc:511] Master 127.11.220.254:45041 requested a full tablet report, sending...
I20260925 02:29:25.816056 26666 ts_manager.cc:194] Registered new tserver with Master: 4592c62e71984bda8343057e444cb3bc (127.11.220.195:40461)
I20260925 02:29:25.816407 26666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:39727
I20260925 02:29:25.817345 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 27001
I20260925 02:29:25.822497 26861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.817628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39178 (local address 127.11.220.254:45041)
0925 02:29:25.817656 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.817658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.817755 (+ 97us) server_negotiation.cc:407] Connection header received
0925 02:29:25.817805 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.817806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.817812 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.817856 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:25.818342 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.818745 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.819501 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.819655 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.819697 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.820672 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:25.820673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:25.820673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:25.820679 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:25.821727 (+ 1048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:25.821999 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:25.822000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:25.822000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:25.822042 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:25.822175 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:25.822175 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:25.822175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:25.822247 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:25.822346 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.822387 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.822437 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260925 02:29:25.823163 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:29:25.825250 26861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.823457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39194 (local address 127.11.220.254:45041)
0925 02:29:25.823501 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.823502 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.823576 (+ 74us) server_negotiation.cc:407] Connection header received
0925 02:29:25.823628 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.823629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.823651 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.823699 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:29:25.823772 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.824136 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.824723 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.824871 (+ 148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.824917 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.825022 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:29:25.825090 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:29:25.825142 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.825168 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.825180 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 02:29:25.825496 26666 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:29:25.826894 26666 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:29:25.828799 26771 tablet_service.cc:1467] Processing CreateTablet for tablet d2b4e6e42a7c484c8e3df348c7e8e392 (DEFAULT_TABLE table=test-table [id=2db5babdbb4e46d08dc12d5e53e6d7f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:25.828990 26771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2b4e6e42a7c484c8e3df348c7e8e392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:25.830420 26747 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: Bootstrap starting.
I20260925 02:29:25.830709 26747 tablet_bootstrap.cc:654] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:25.831156 26747 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: No bootstrap required, opened a new log
I20260925 02:29:25.831216 26747 ts_tablet_manager.cc:1397] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260925 02:29:25.831552 26747 raft_consensus.cc:348] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:25.831641 26747 raft_consensus.cc:374] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:25.831693 26747 raft_consensus.cc:729] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d843c7dd44b48a3a103e6b98d953fda, State: Initialized, Role: FOLLOWER
I20260925 02:29:25.831733 26747 consensus_queue.cc:260] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:25.831837 26747 ts_tablet_manager.cc:1428] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 02:29:25.832358 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.828529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40461 (local address 127.0.0.1:35478)
0925 02:29:25.828688 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.828697 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.828738 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.829179 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.829179 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.829181 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:25.829225 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.829226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.830218 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.830218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.831232 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.831234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.832118 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.832118 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.832185 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.832202 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.832250 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.832294 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":12,"thread_start_us":58,"threads_started":1}
I20260925 02:29:25.832396 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.828602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.11.220.194:33493)
0925 02:29:25.828792 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.828797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.828805 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:29:25.828836 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.828839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.828857 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.828939 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:25.829078 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.830057 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.831322 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.831969 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.832030 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.832086 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.832274 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.832319 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":81,"threads_started":1}
I20260925 02:29:25.832456 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.828482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:33493 (local address 127.0.0.1:33672)
0925 02:29:25.828515 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.828525 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.828572 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.828943 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.828944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.828945 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:25.828981 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.828982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.830161 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.830162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.831232 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.831234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.832118 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.832118 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.832190 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.832204 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.832248 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.832294 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":54}
I20260925 02:29:25.832643 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.828691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.11.220.195:40461)
0925 02:29:25.829047 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.829050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.829058 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:29:25.829083 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.829085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.829096 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.829180 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:25.829271 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.830104 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.831322 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.831972 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.832047 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.832086 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.832529 (+ 443us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.832576 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":67,"threads_started":1}
I20260925 02:29:25.833148 26931 tablet_service.cc:1467] Processing CreateTablet for tablet d2b4e6e42a7c484c8e3df348c7e8e392 (DEFAULT_TABLE table=test-table [id=2db5babdbb4e46d08dc12d5e53e6d7f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:25.833241 27065 tablet_service.cc:1467] Processing CreateTablet for tablet d2b4e6e42a7c484c8e3df348c7e8e392 (DEFAULT_TABLE table=test-table [id=2db5babdbb4e46d08dc12d5e53e6d7f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:25.833422 26931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2b4e6e42a7c484c8e3df348c7e8e392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:25.833539 27065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2b4e6e42a7c484c8e3df348c7e8e392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:25.835916 27141 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Bootstrap starting.
I20260925 02:29:25.836024 27142 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Bootstrap starting.
I20260925 02:29:25.836499 27141 tablet_bootstrap.cc:654] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:25.836583 27142 tablet_bootstrap.cc:654] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:25.836988 27141 log.cc:824] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:25.837020 27142 log.cc:824] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:25.837617 27141 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: No bootstrap required, opened a new log
I20260925 02:29:25.837636 27142 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: No bootstrap required, opened a new log
I20260925 02:29:25.837687 27142 ts_tablet_manager.cc:1397] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 02:29:25.837694 27141 ts_tablet_manager.cc:1397] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 02:29:25.838829 27142 raft_consensus.cc:348] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:25.838940 27142 raft_consensus.cc:374] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:25.838852 27141 raft_consensus.cc:348] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:25.838961 27142 raft_consensus.cc:729] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4477257e1a264b8c9a6c0aca0ec74ffd, State: Initialized, Role: FOLLOWER
I20260925 02:29:25.838982 27141 raft_consensus.cc:374] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:25.839028 27141 raft_consensus.cc:729] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4592c62e71984bda8343057e444cb3bc, State: Initialized, Role: FOLLOWER
I20260925 02:29:25.839071 27142 consensus_queue.cc:260] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:25.839149 27141 consensus_queue.cc:260] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:25.839355 27142 ts_tablet_manager.cc:1428] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:29:25.839427 27141 ts_tablet_manager.cc:1428] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:29:25.839471 27132 heartbeater.cc:503] Master 127.11.220.254:45041 was elected leader, sending a full tablet report...
I20260925 02:29:25.839547 26998 heartbeater.cc:503] Master 127.11.220.254:45041 was elected leader, sending a full tablet report...
I20260925 02:29:25.855494 27146 raft_consensus.cc:482] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:29:25.855581 27146 raft_consensus.cc:504] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:25.855913 27146 leader_election.cc:288] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4477257e1a264b8c9a6c0aca0ec74ffd (127.11.220.194:33493), 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287)
I20260925 02:29:25.858199 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.856040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:33493 (local address 127.11.220.195:51261)
0925 02:29:25.856088 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.856098 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.856138 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.856263 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.856264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.856265 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:25.856310 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.856311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.856867 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.856867 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.857483 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.857484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.857969 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.857969 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.858040 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.858043 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.858111 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.858141 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":3}
I20260925 02:29:25.858199 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.856136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:51261 (local address 127.11.220.194:33493)
0925 02:29:25.856174 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.856176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.856183 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:29:25.856203 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.856203 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.856210 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.856261 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:25.856356 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.856792 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.857564 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.857896 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.857970 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.857976 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.858116 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.858152 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:29:25.858453 26952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2b4e6e42a7c484c8e3df348c7e8e392" candidate_uuid: "4592c62e71984bda8343057e444cb3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" is_pre_election: true
I20260925 02:29:25.858561 26866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.856192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:60851 (local address 127.11.220.193:44287)
0925 02:29:25.856233 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.856234 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.856275 (+ 41us) server_negotiation.cc:407] Connection header received
0925 02:29:25.856328 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.856329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.856345 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.856394 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:25.856530 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.856956 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.857778 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.858130 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.858208 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.858259 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.858459 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.858507 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:29:25.858559 27147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.856126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44287 (local address 127.11.220.195:60851)
0925 02:29:25.856237 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.856244 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:25.856285 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:25.856396 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:25.856396 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:25.856397 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:25.856478 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.856479 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.857042 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.857043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.857691 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:25.857692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.858208 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.858209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:25.858386 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.858390 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:29:25.858435 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 02:29:25.858467 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260925 02:29:25.858570 26952 raft_consensus.cc:2454] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4592c62e71984bda8343057e444cb3bc in term 0.
I20260925 02:29:25.858788 26811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2b4e6e42a7c484c8e3df348c7e8e392" candidate_uuid: "4592c62e71984bda8343057e444cb3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d843c7dd44b48a3a103e6b98d953fda" is_pre_election: true
I20260925 02:29:25.858913 26811 raft_consensus.cc:2454] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4592c62e71984bda8343057e444cb3bc in term 0.
I20260925 02:29:25.859067 27019 leader_election.cc:302] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4477257e1a264b8c9a6c0aca0ec74ffd, 4592c62e71984bda8343057e444cb3bc; no voters:
I20260925 02:29:25.859196 27146 raft_consensus.cc:2781] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:29:25.859256 27146 raft_consensus.cc:482] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:29:25.859283 27146 raft_consensus.cc:3037] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:25.859843 27146 raft_consensus.cc:504] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:25.859952 27146 leader_election.cc:288] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [CANDIDATE]: Term 1 election: Requested vote from peers 4477257e1a264b8c9a6c0aca0ec74ffd (127.11.220.194:33493), 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287)
I20260925 02:29:25.860157 26811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2b4e6e42a7c484c8e3df348c7e8e392" candidate_uuid: "4592c62e71984bda8343057e444cb3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d843c7dd44b48a3a103e6b98d953fda"
I20260925 02:29:25.860188 26952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2b4e6e42a7c484c8e3df348c7e8e392" candidate_uuid: "4592c62e71984bda8343057e444cb3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd"
I20260925 02:29:25.860275 26811 raft_consensus.cc:3037] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:25.860270 26952 raft_consensus.cc:3037] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:25.860925 26811 raft_consensus.cc:2454] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4592c62e71984bda8343057e444cb3bc in term 1.
I20260925 02:29:25.860956 26952 raft_consensus.cc:2454] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4592c62e71984bda8343057e444cb3bc in term 1.
I20260925 02:29:25.861110 27019 leader_election.cc:302] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d843c7dd44b48a3a103e6b98d953fda, 4592c62e71984bda8343057e444cb3bc; no voters:
I20260925 02:29:25.861232 27146 raft_consensus.cc:2781] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:29:25.861377 27146 raft_consensus.cc:686] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 1 LEADER]: Becoming Leader. State: Replica: 4592c62e71984bda8343057e444cb3bc, State: Running, Role: LEADER
I20260925 02:29:25.861461 27146 consensus_queue.cc:237] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:25.862105 26665 catalog_manager.cc:5495] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 4592c62e71984bda8343057e444cb3bc (127.11.220.195). New cstate: current_term: 1 leader_uuid: "4592c62e71984bda8343057e444cb3bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } health_report { overall_health: UNKNOWN } } }
I20260925 02:29:25.867153 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.865125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35480 (local address 127.11.220.195:40461)
0925 02:29:25.865181 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:25.865183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:25.865192 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:29:25.865251 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:25.865252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:25.865265 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:25.865317 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:29:25.865429 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.865843 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.866468 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:25.866652 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:25.866718 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:25.866864 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:29:25.866943 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:29:25.867015 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:25.867060 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 02:29:25.867076 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:29:25.868409 26952 raft_consensus.cc:1261] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Refusing update from remote peer 4592c62e71984bda8343057e444cb3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:29:25.868439 26811 raft_consensus.cc:1261] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Refusing update from remote peer 4592c62e71984bda8343057e444cb3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:29:25.868711 27148 consensus_queue.cc:1035] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:29:25.868821 27146 consensus_queue.cc:1035] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:29:25.870781 27151 mvcc.cc:205] Tried to move back new op lower bound from 7333082586595209216 to 7333082586569117696. Current Snapshot: MvccSnapshot[applied={T|T < 7333082586595209216}]
I20260925 02:29:25.870994 27150 mvcc.cc:205] Tried to move back new op lower bound from 7333082586595209216 to 7333082586569117696. Current Snapshot: MvccSnapshot[applied={T|T < 7333082586595209216 or (T in {7333082586595209216})}]
I20260925 02:29:25.871860 27152 mvcc.cc:205] Tried to move back new op lower bound from 7333082586595209216 to 7333082586569117696. Current Snapshot: MvccSnapshot[applied={T|T < 7333082586595209216 or (T in {7333082586595209216})}]
I20260925 02:29:25.872625 26665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:29:25.872706 26665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:29:25.873270 26665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Sending DeleteTablet for 3 replicas of tablet d2b4e6e42a7c484c8e3df348c7e8e392
I20260925 02:29:25.873595 27065 tablet_service.cc:1514] Processing DeleteTablet for tablet d2b4e6e42a7c484c8e3df348c7e8e392 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:25 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:35478
I20260925 02:29:25.873594 26931 tablet_service.cc:1514] Processing DeleteTablet for tablet d2b4e6e42a7c484c8e3df348c7e8e392 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:25 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:33672
I20260925 02:29:25.873777 26771 tablet_service.cc:1514] Processing DeleteTablet for tablet d2b4e6e42a7c484c8e3df348c7e8e392 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:25 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:52160
I20260925 02:29:25.873894 26867 tablet_replica.cc:330] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: stopping tablet replica
I20260925 02:29:25.873921 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 26705
I20260925 02:29:25.873950 26867 raft_consensus.cc:2229] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:29:25.873991 26867 raft_consensus.cc:2258] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 02:29:25.879621 26651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:44287 (error 108)
I20260925 02:29:25.879820 27160 tablet_replica.cc:330] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: stopping tablet replica
I20260925 02:29:25.879908 27161 tablet_replica.cc:330] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: stopping tablet replica
I20260925 02:29:25.879916 27160 raft_consensus.cc:2229] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:29:25.880020 27161 raft_consensus.cc:2229] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:29:25.880028 27160 raft_consensus.cc:2258] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:29:25.880105 27161 raft_consensus.cc:2258] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:29:25.880179 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 26863
I20260925 02:29:25.880190 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.880050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44287 (local address 127.0.0.1:52170)
0925 02:29:25.880119 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.880165 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:44287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260925 02:29:25.880370 26651 catalog_manager.cc:4584] TS 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d2b4e6e42a7c484c8e3df348c7e8e392: Network error: Client connection negotiation failed: client connection to 127.11.220.193:44287: connect: Connection refused (error 111)
I20260925 02:29:25.881911 27160 ts_tablet_manager.cc:1905] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:29:25.883353 27160 ts_tablet_manager.cc:1918] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:29:25.883440 27160 log.cc:1196] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/d2b4e6e42a7c484c8e3df348c7e8e392
I20260925 02:29:25.883700 27160 ts_tablet_manager.cc:1939] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Deleting consensus metadata
I20260925 02:29:25.884059 26652 catalog_manager.cc:4841] TS 4592c62e71984bda8343057e444cb3bc (127.11.220.195:40461): tablet d2b4e6e42a7c484c8e3df348c7e8e392 (table test-table [id=2db5babdbb4e46d08dc12d5e53e6d7f4]) successfully deleted
I20260925 02:29:25.885699 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 27001
I20260925 02:29:25.886296 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:25.885766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:33493 (local address 127.0.0.1:33682)
0925 02:29:25.886238 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:25.886265 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.194:33493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456}
I20260925 02:29:25.890638 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 26632
I20260925 02:29:25.895617 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:45041
--webserver_interface=127.11.220.254
--webserver_port=37089
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:45041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:25.967618 27163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:25.967778 27163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:25.967798 27163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:25.968896 27163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:29:25.968931 27163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:25.968945 27163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:25.968972 27163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:29:25.969007 27163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:29:25.970332 27163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:45041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:45041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=37089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:25.970569 27163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:25.970775 27163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:25.975169 27168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:25.975167 27169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:25.975167 27171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:25.975389 27163 server_base.cc:1034] running on GCE node
I20260925 02:29:25.975605 27163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:25.975828 27163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:25.976958 27163 hybrid_clock.cc:648] HybridClock initialized: now 1790303365976949 us; error 27 us; skew 500 ppm
Sep 25 02:29:25 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303365, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:25.979036 27163 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:29:25.979403 27163 webserver.cc:460] Webserver started at http://127.11.220.254:37089/ using document root <none> and password file <none>
I20260925 02:29:25.979619 27163 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:25.979686 27163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:25.981003 27163 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:25.981787 27179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:25.981956 27163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:25.982050 27163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "80210798db0aa4b9bc588146f1f2cbde"
server_key_iv: "663194cab71e144347a12ef886528049"
server_key_version: "encryptionkey@0"
I20260925 02:29:25.982339 27163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:25.999449 27163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:26.001281 27163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:26.001433 27163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:26.005066 27163 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:45041
I20260925 02:29:26.005103 27232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:45041 every 8 connection(s)
I20260925 02:29:26.005446 27163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:29:26.006862 27233 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:29:26.007480 27233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Bootstrap starting.
I20260925 02:29:26.009968 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 27163
I20260925 02:29:26.010223 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:44287
--local_ip_for_outbound_sockets=127.11.220.193
--tserver_master_addrs=127.11.220.254:45041
--webserver_port=33431
--webserver_interface=127.11.220.193
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:29:26.016136 27233 log.cc:824] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:26.018312 27233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:29:26.018608 27233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4: Bootstrap complete.
I20260925 02:29:26.020769 27233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:26.020984 27233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d8f2badcbb4f4ea204aecba8cdcdd4, State: Initialized, Role: FOLLOWER
I20260925 02:29:26.021166 27233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:26.021255 27233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:29:26.021303 27233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:29:26.021346 27233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 2 FOLLOWER]: Advancing to term 3
I20260925 02:29:26.022194 27233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:26.022302 27233 leader_election.cc:302] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88d8f2badcbb4f4ea204aecba8cdcdd4; no voters:
I20260925 02:29:26.022467 27233 leader_election.cc:288] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 02:29:26.022526 27237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 02:29:26.022723 27237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [term 3 LEADER]: Becoming Leader. State: Replica: 88d8f2badcbb4f4ea204aecba8cdcdd4, State: Running, Role: LEADER
I20260925 02:29:26.022754 27233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:29:26.022821 27237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } }
I20260925 02:29:26.023067 27237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88d8f2badcbb4f4ea204aecba8cdcdd4. Latest consensus state: current_term: 3 leader_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } } }
I20260925 02:29:26.023136 27237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:26.023073 27238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d8f2badcbb4f4ea204aecba8cdcdd4" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45041 } } }
I20260925 02:29:26.023327 27238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d8f2badcbb4f4ea204aecba8cdcdd4 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:26.023716 27243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:29:26.024420 27243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:29:26.024531 27243 catalog_manager.cc:1259] Loaded cluster ID: 8390e279021a4f97be0989995c836bc5
I20260925 02:29:26.024583 27243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 02:29:26.024938 27243 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc0e48889ad google::LogMessage::Fail() at ??:0
@ 0x7fc0e488b88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc0e48884de google::LogMessage::Flush() at ??:0
@ 0x7fc0e4889a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc0ec7ae0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc0ec7b78b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc0e67560d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc0e674b954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc0e75af6db start_thread at ??:0
@ 0x7fc0e338661f clone at ??:0
W20260925 02:29:26.099848 27235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:26.100028 27235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:26.100049 27235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:26.101188 27235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:26.101224 27235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:26.101248 27235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:29:26.102517 27235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:44287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=33431
--tserver_master_addrs=127.11.220.254:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:26.102718 27235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:26.102936 27235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:26.108426 27270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:26.108507 27235 server_base.cc:1034] running on GCE node
W20260925 02:29:26.108433 27272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:26.108671 27269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:26.108851 27235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:26.109093 27235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:26.110245 27235 hybrid_clock.cc:648] HybridClock initialized: now 1790303366110236 us; error 23 us; skew 500 ppm
Sep 25 02:29:26 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:26.112677 27235 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:29:26.113097 27235 webserver.cc:460] Webserver started at http://127.11.220.193:33431/ using document root <none> and password file <none>
I20260925 02:29:26.113308 27235 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:26.113371 27235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:26.114567 27235 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:26.115242 27279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:26.115409 27235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:26.115500 27235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "3d843c7dd44b48a3a103e6b98d953fda"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "bbbdfeda9f6b1c9ddc748f0e4c88b264"
server_key_iv: "3222fc11d33bc4433241ed8627ed0557"
server_key_version: "encryptionkey@0"
I20260925 02:29:26.115891 27235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:26.147218 27235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:26.149195 27235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:26.149374 27235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:26.149590 27235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:26.149972 27286 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 02:29:26.151734 27235 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 02:29:26.151798 27235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 02:29:26.151840 27235 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 02:29:26.152413 27235 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 02:29:26.152489 27235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:26.152551 27286 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: Bootstrap starting.
I20260925 02:29:26.158640 27235 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:44287
I20260925 02:29:26.158756 27394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:44287 every 8 connection(s)
I20260925 02:29:26.158993 27235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:29:26.162782 27286 log.cc:824] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:26.165052 27286 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:29:26.165375 27286 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: Bootstrap complete.
I20260925 02:29:26.165436 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 27235
I20260925 02:29:26.165679 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:33493
--local_ip_for_outbound_sockets=127.11.220.194
--tserver_master_addrs=127.11.220.254:45041
--webserver_port=42443
--webserver_interface=127.11.220.194
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:29:26.166515 27286 ts_tablet_manager.cc:1397] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260925 02:29:26.167685 27286 raft_consensus.cc:348] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:26.167987 27286 raft_consensus.cc:729] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d843c7dd44b48a3a103e6b98d953fda, State: Initialized, Role: FOLLOWER
I20260925 02:29:26.168107 27286 consensus_queue.cc:260] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:26.168390 27286 ts_tablet_manager.cc:1428] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
W20260925 02:29:26.197916 27397 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:29:26.159612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:44453)
0925 02:29:26.159812 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.159842 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:26.160216 (+ 374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:26.197737 (+ 37521us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":124,"threads_started":1}
W20260925 02:29:26.198134 27395 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:45041 (0 consecutive failures): Network error: Failed to ping master at 127.11.220.254:45041: Client connection negotiation failed: client connection to 127.11.220.254:45041: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 02:29:26.198511 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:26.198388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:42091)
0925 02:29:26.198435 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.198488 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 02:29:26.198904 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:26.198818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:34153)
0925 02:29:26.198860 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.198886 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 02:29:26.199011 27395 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:29:26.240352 27399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:26.240517 27399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:26.240538 27399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:26.241573 27399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:26.241608 27399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:26.241653 27399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:29:26.242974 27399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:33493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=42443
--tserver_master_addrs=127.11.220.254:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:26.243141 27399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:26.243342 27399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:29:26.247767 27399 server_base.cc:1034] running on GCE node
W20260925 02:29:26.247751 27405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:26.247749 27408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:26.247740 27406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:26.248109 27399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:26.248353 27399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:26.249488 27399 hybrid_clock.cc:648] HybridClock initialized: now 1790303366249483 us; error 30 us; skew 500 ppm
Sep 25 02:29:26 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:26.251749 27399 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:29:26.252154 27399 webserver.cc:460] Webserver started at http://127.11.220.194:42443/ using document root <none> and password file <none>
I20260925 02:29:26.252329 27399 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:26.252367 27399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:26.253564 27399 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:26.254118 27415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:26.254274 27399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:26.254354 27399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "4477257e1a264b8c9a6c0aca0ec74ffd"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "b4c1d2a5dc1cd7fbc7ad3c8f100672ff"
server_key_iv: "147bae7163ba22b9bec845962045597c"
server_key_version: "encryptionkey@0"
I20260925 02:29:26.254662 27399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:26.262684 27399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:26.264237 27399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:26.264367 27399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:26.264601 27399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:26.264994 27422 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 02:29:26.266232 27399 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 02:29:26.266283 27399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260925 02:29:26.266338 27399 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 02:29:26.266875 27399 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 02:29:26.266927 27399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 02:29:26.266976 27422 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Bootstrap starting.
I20260925 02:29:26.273528 27399 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:33493
I20260925 02:29:26.273630 27533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:33493 every 8 connection(s)
I20260925 02:29:26.273888 27399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:29:26.274883 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:26.274404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:39493)
0925 02:29:26.274572 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.274694 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
W20260925 02:29:26.275084 27534 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:45041 (0 consecutive failures): Network error: Failed to ping master at 127.11.220.254:45041: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
I20260925 02:29:26.275408 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:26.275285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:49087)
0925 02:29:26.275342 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.275377 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 02:29:26.275705 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:26.275631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:58001)
0925 02:29:26.275658 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.275682 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260925 02:29:26.275857 27534 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 02:29:26.276151 27422 log.cc:824] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:26.277921 27422 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:29:26.278180 27422 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Bootstrap complete.
I20260925 02:29:26.279135 27422 ts_tablet_manager.cc:1397] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.009s
I20260925 02:29:26.280208 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 27399
I20260925 02:29:26.280480 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:40461
--local_ip_for_outbound_sockets=127.11.220.195
--tserver_master_addrs=127.11.220.254:45041
--webserver_port=45425
--webserver_interface=127.11.220.195
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:29:26.280380 27422 raft_consensus.cc:348] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:26.280753 27422 raft_consensus.cc:729] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4477257e1a264b8c9a6c0aca0ec74ffd, State: Initialized, Role: FOLLOWER
I20260925 02:29:26.280879 27422 consensus_queue.cc:260] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:26.281108 27422 ts_tablet_manager.cc:1428] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
I20260925 02:29:26.281291 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:26.281219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:38523)
0925 02:29:26.281246 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.281268 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260925 02:29:26.281545 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:26.281461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:59095)
0925 02:29:26.281508 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.281527 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 02:29:26.353077 27539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:26.353238 27539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:26.353258 27539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:26.354293 27539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:26.354328 27539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:26.354375 27539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:29:26.355741 27539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:40461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=45425
--tserver_master_addrs=127.11.220.254:45041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:26.355955 27539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:26.356165 27539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:26.360654 27544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:26.360648 27545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:26.360632 27547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:26.360843 27539 server_base.cc:1034] running on GCE node
I20260925 02:29:26.361011 27539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:26.361188 27539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:26.362331 27539 hybrid_clock.cc:648] HybridClock initialized: now 1790303366362322 us; error 27 us; skew 500 ppm
Sep 25 02:29:26 dist-test-slave-dn4n krb5kdc[26077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:26.364614 27539 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:29:26.365020 27539 webserver.cc:460] Webserver started at http://127.11.220.195:45425/ using document root <none> and password file <none>
I20260925 02:29:26.365242 27539 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:26.365284 27539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:26.366459 27539 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:26.366966 27554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:26.367118 27539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:26.367206 27539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "4592c62e71984bda8343057e444cb3bc"
format_stamp: "Formatted at 2026-09-25 02:29:24 on dist-test-slave-dn4n"
server_key: "075d50cfd0cb34e34e0e45e63932ec29"
server_key_iv: "c710bf6a2efb5a26b53522565a01ca60"
server_key_version: "encryptionkey@0"
I20260925 02:29:26.367472 27539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:26.379038 27539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:26.380604 27539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:26.380731 27539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:26.380914 27539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:26.381182 27539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:26.381229 27539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:26.381264 27539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:26.381287 27539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:26.386533 27539 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:40461
I20260925 02:29:26.386572 27669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:40461 every 8 connection(s)
I20260925 02:29:26.386873 27539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:29:26.387722 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:26.387411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:55977)
0925 02:29:26.387539 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.387604 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
W20260925 02:29:26.388068 27670 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:45041 (0 consecutive failures): Network error: Failed to ping master at 127.11.220.254:45041: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
I20260925 02:29:26.388310 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:26.388232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:45913)
0925 02:29:26.388271 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.388294 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 02:29:26.388605 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:26.388523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:55433)
0925 02:29:26.388559 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:26.388587 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260925 02:29:26.388757 27670 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 02:29:26.395048 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 27539
I20260925 02:29:27.199606 27674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:27.199372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:53803)
0925 02:29:27.199497 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:27.199542 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260925 02:29:27.282122 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:27.281939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:53735)
0925 02:29:27.282048 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:27.282078 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260925 02:29:27.389158 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:27.388988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:40747)
0925 02:29:27.389087 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:27.389115 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":48,"threads_started":1}
I20260925 02:29:27.485071 27677 raft_consensus.cc:482] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:29:27.485179 27677 raft_consensus.cc:504] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:27.485464 27677 leader_election.cc:288] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4592c62e71984bda8343057e444cb3bc (127.11.220.195:40461), 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287)
Sep 25 02:29:27 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.195@KRBTEST.COM
Sep 25 02:29:27 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:29:27.494694 27680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:27.485804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:40435 (local address 127.11.220.193:44287)
0925 02:29:27.486006 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:27.486010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:27.486016 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:29:27.486073 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:27.486074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:27.486091 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:27.486174 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:27.486867 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:27.487659 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:27.488635 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:27.489018 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:27.489179 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:27.491681 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:27.491688 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:27.491689 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:27.491698 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:27.493181 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:27.493540 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:27.493543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:27.493543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:27.493627 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:27.493884 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:27.493885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:27.493885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:27.494111 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:27.494210 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:27.494479 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 02:29:27.494597 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
I20260925 02:29:27.494694 27675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:27.485582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40461 (local address 127.11.220.194:59259)
0925 02:29:27.485634 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:27.485677 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:27.485862 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:27.486476 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:27.486477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:27.486695 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:27.486762 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:27.486767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:27.487748 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:27.487749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:27.488548 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:27.488551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:27.489164 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:27.489164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:27.489250 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:27.489799 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:27.489806 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:27.491301 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:27.493149 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:27.493155 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:27.493158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:27.493461 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:27.493695 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:27.493696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:27.493697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:27.493821 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:27.494181 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:27.494188 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:27.494310 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 02:29:27.494407 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 02:29:27.494585 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":3}
I20260925 02:29:27.494719 27679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:27.485685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:59259 (local address 127.11.220.195:40461)
0925 02:29:27.485846 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:27.485856 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:27.485898 (+ 42us) server_negotiation.cc:407] Connection header received
0925 02:29:27.486318 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:27.486320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:27.486350 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:27.486496 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:27.486874 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:27.487650 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:27.488642 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:27.489062 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:27.489183 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:27.491400 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:27.491404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:27.491404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:27.491415 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:27.493074 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:27.493544 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:27.493547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:27.493547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:27.493629 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:27.493885 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:27.493886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:27.493886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:27.494113 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:27.494197 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:27.494473 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 02:29:27.494607 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":56,"threads_started":1}
I20260925 02:29:27.494789 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:27.485607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44287 (local address 127.11.220.194:40435)
0925 02:29:27.485723 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:27.485732 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:27.485874 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:27.486211 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:27.486216 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:27.486599 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:27.486762 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:27.486767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:27.487757 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:27.487758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:27.488550 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:27.488552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:27.489146 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:27.489147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:27.489250 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:27.489851 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:27.489855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:27.491583 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:27.493260 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:27.493261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:27.493261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:27.493462 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:27.493695 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:27.493696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:27.493696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:27.493821 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:27.494208 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:27.494208 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:27.494309 (+ 101us) client_negotiation.cc:769] Sending connection context
0925 02:29:27.494407 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 02:29:27.494663 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":171,"thread_start_us":60,"threads_started":1}
W20260925 02:29:27.495410 27419 leader_election.cc:341] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4592c62e71984bda8343057e444cb3bc (127.11.220.195:40461): Not found: Tablet not found: d2b4e6e42a7c484c8e3df348c7e8e392
I20260925 02:29:27.495862 27348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2b4e6e42a7c484c8e3df348c7e8e392" candidate_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3d843c7dd44b48a3a103e6b98d953fda" is_pre_election: true
I20260925 02:29:27.495994 27348 raft_consensus.cc:2454] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4477257e1a264b8c9a6c0aca0ec74ffd in term 1.
I20260925 02:29:27.496234 27418 leader_election.cc:302] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d843c7dd44b48a3a103e6b98d953fda, 4477257e1a264b8c9a6c0aca0ec74ffd; no voters: 4592c62e71984bda8343057e444cb3bc
I20260925 02:29:27.496345 27677 raft_consensus.cc:2781] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 02:29:27.496413 27677 raft_consensus.cc:482] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:29:27.496474 27677 raft_consensus.cc:3037] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:29:27.497480 27677 raft_consensus.cc:504] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:27.497619 27677 leader_election.cc:288] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [CANDIDATE]: Term 2 election: Requested vote from peers 4592c62e71984bda8343057e444cb3bc (127.11.220.195:40461), 3d843c7dd44b48a3a103e6b98d953fda (127.11.220.193:44287)
W20260925 02:29:27.497879 27419 leader_election.cc:341] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 4592c62e71984bda8343057e444cb3bc (127.11.220.195:40461): Not found: Tablet not found: d2b4e6e42a7c484c8e3df348c7e8e392
I20260925 02:29:27.497846 27348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2b4e6e42a7c484c8e3df348c7e8e392" candidate_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3d843c7dd44b48a3a103e6b98d953fda"
I20260925 02:29:27.497941 27348 raft_consensus.cc:3037] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:29:27.498824 27348 raft_consensus.cc:2454] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4477257e1a264b8c9a6c0aca0ec74ffd in term 2.
I20260925 02:29:27.498982 27418 leader_election.cc:302] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d843c7dd44b48a3a103e6b98d953fda, 4477257e1a264b8c9a6c0aca0ec74ffd; no voters: 4592c62e71984bda8343057e444cb3bc
I20260925 02:29:27.499073 27677 raft_consensus.cc:2781] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 2 FOLLOWER]: Leader election won for term 2
I20260925 02:29:27.499162 27677 raft_consensus.cc:686] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [term 2 LEADER]: Becoming Leader. State: Replica: 4477257e1a264b8c9a6c0aca0ec74ffd, State: Running, Role: LEADER
I20260925 02:29:27.499250 27677 consensus_queue.cc:237] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:27.499511 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:27.499364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:46485)
0925 02:29:27.499452 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:27.499481 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260925 02:29:27.938377 27419 consensus_peers.cc:487] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd -> Peer 4592c62e71984bda8343057e444cb3bc (127.11.220.195:40461): Couldn't send request to peer 4592c62e71984bda8343057e444cb3bc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d2b4e6e42a7c484c8e3df348c7e8e392. This is attempt 1: this message will repeat every 5th retry.
I20260925 02:29:28.087838 27348 raft_consensus.cc:1261] T d2b4e6e42a7c484c8e3df348c7e8e392 P 3d843c7dd44b48a3a103e6b98d953fda [term 2 FOLLOWER]: Refusing update from remote peer 4477257e1a264b8c9a6c0aca0ec74ffd: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260925 02:29:28.088207 27677 consensus_queue.cc:1035] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260925 02:29:28.089068 27685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:28.088919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:58371)
0925 02:29:28.088994 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:28.089034 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64,"thread_start_us":38,"threads_started":1}
I20260925 02:29:28.089947 27685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:28.089869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:35999)
0925 02:29:28.089897 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:28.089928 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260925 02:29:28.089951 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:28.089780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:54815)
0925 02:29:28.089877 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:28.089907 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":44,"threads_started":1}
I20260925 02:29:28.389673 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:28.389516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:33659)
0925 02:29:28.389612 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:28.389641 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":50,"threads_started":1}
I20260925 02:29:28.467620 27691 ts_tablet_manager.cc:927] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Initiating tablet copy from peer 4477257e1a264b8c9a6c0aca0ec74ffd (127.11.220.194:33493)
I20260925 02:29:28.467818 27691 tablet_copy_client.cc:316] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: tablet copy: Beginning tablet copy session from remote peer at address 127.11.220.194:33493
Sep 25 02:29:28 dist-test-slave-dn4n krb5kdc[26077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303366, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.194@KRBTEST.COM
I20260925 02:29:28.474705 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:28.468189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:50423 (local address 127.11.220.194:33493)
0925 02:29:28.468341 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:28.468345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:28.468354 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:29:28.468377 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:28.468378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:28.468390 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:28.468489 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:28.468897 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:28.469591 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:28.470293 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:28.470565 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:28.470660 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:28.472546 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:28.472553 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:28.472553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:28.472564 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:28.473772 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:28.474100 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:28.474103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:28.474103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:28.474151 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:28.474300 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:28.474301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:28.474301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:28.474419 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:28.474474 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:28.474592 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 02:29:28.474644 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260925 02:29:28.474720 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:28.468061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:33493 (local address 127.11.220.195:50423)
0925 02:29:28.468119 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:28.468139 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:28.468208 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:28.468494 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:28.468499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:28.468759 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:28.468831 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:28.468832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:28.469655 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:28.469656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:28.470196 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:28.470198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:28.470660 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:28.470661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:28.470711 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:28.471176 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:28.471180 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:28.472434 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:28.473838 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:28.473844 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:28.473846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:28.474034 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:28.474204 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:28.474204 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:28.474204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:28.474253 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:28.474468 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:28.474473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:28.474514 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 02:29:28.474570 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 02:29:28.474652 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 02:29:28.475126 27508 tablet_copy_service.cc:140] P 4477257e1a264b8c9a6c0aca0ec74ffd: Received BeginTabletCopySession request for tablet d2b4e6e42a7c484c8e3df348c7e8e392 from peer 4592c62e71984bda8343057e444cb3bc ({username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:50423)
I20260925 02:29:28.475225 27508 tablet_copy_service.cc:161] P 4477257e1a264b8c9a6c0aca0ec74ffd: Beginning new tablet copy session on tablet d2b4e6e42a7c484c8e3df348c7e8e392 from peer 4592c62e71984bda8343057e444cb3bc at {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:50423: session id = 4592c62e71984bda8343057e444cb3bc-d2b4e6e42a7c484c8e3df348c7e8e392
I20260925 02:29:28.475657 27508 tablet_copy_source_session.cc:205] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4477257e1a264b8c9a6c0aca0ec74ffd: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 02:29:28.476567 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2b4e6e42a7c484c8e3df348c7e8e392. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:28.479028 27691 tablet_copy_client.cc:791] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: tablet copy: Starting download of 0 data blocks...
I20260925 02:29:28.479176 27691 tablet_copy_client.cc:655] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: tablet copy: Starting download of 1 WAL segments...
I20260925 02:29:28.479882 27691 tablet_copy_client.cc:531] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 02:29:28.480540 27691 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Bootstrap starting.
I20260925 02:29:28.481705 27691 log.cc:824] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:28.483691 27691 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:29:28.483848 27691 tablet_bootstrap.cc:492] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Bootstrap complete.
I20260925 02:29:28.483934 27691 ts_tablet_manager.cc:1397] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 02:29:28.484788 27691 raft_consensus.cc:348] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:28.484910 27691 raft_consensus.cc:729] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4592c62e71984bda8343057e444cb3bc, State: Initialized, Role: FOLLOWER
I20260925 02:29:28.484983 27691 consensus_queue.cc:260] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4477257e1a264b8c9a6c0aca0ec74ffd" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33493 } } peers { permanent_uuid: "4592c62e71984bda8343057e444cb3bc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40461 } } peers { permanent_uuid: "3d843c7dd44b48a3a103e6b98d953fda" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44287 } }
I20260925 02:29:28.485157 27691 ts_tablet_manager.cc:1428] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20260925 02:29:28.485399 27508 tablet_copy_service.cc:342] P 4477257e1a264b8c9a6c0aca0ec74ffd: Request end of tablet copy session 4592c62e71984bda8343057e444cb3bc-d2b4e6e42a7c484c8e3df348c7e8e392 received from {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:50423
I20260925 02:29:28.485440 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:28.485357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:56335)
0925 02:29:28.485387 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:28.485415 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260925 02:29:28.485450 27508 tablet_copy_service.cc:433] P 4477257e1a264b8c9a6c0aca0ec74ffd: ending tablet copy session 4592c62e71984bda8343057e444cb3bc-d2b4e6e42a7c484c8e3df348c7e8e392 on tablet d2b4e6e42a7c484c8e3df348c7e8e392 with peer 4592c62e71984bda8343057e444cb3bc
I20260925 02:29:29.032712 27624 raft_consensus.cc:1203] T d2b4e6e42a7c484c8e3df348c7e8e392 P 4592c62e71984bda8343057e444cb3bc [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260925 02:29:29.033196 27697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:29.032567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:56891)
0925 02:29:29.033105 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:29.033150 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":45,"threads_started":1}
I20260925 02:29:29.033254 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:29.033095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:52587)
0925 02:29:29.033179 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:29.033210 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":47,"threads_started":1}
I20260925 02:29:29.090476 27700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:29.090302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:53483)
0925 02:29:29.090400 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:29.090432 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":47,"threads_started":1}
I20260925 02:29:30.033711 27701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:30.033501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:60969)
0925 02:29:30.033638 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:30.033669 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260925 02:29:30.033855 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:30.033661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:33257)
0925 02:29:30.033763 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:30.033806 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":59,"threads_started":1}
I20260925 02:29:30.091073 27703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:30.090884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:42295)
0925 02:29:30.090996 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:30.091028 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260925 02:29:31.034272 27704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:31.034096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:42021)
0925 02:29:31.034193 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:31.034219 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":58,"threads_started":1}
I20260925 02:29:31.034379 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:31.034207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:44897)
0925 02:29:31.034300 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:31.034330 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":52,"threads_started":1}
I20260925 02:29:31.091636 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:31.091456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:58915)
0925 02:29:31.091558 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:31.091587 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260925 02:29:32.035018 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:32.034825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:44029)
0925 02:29:32.034951 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:32.034977 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260925 02:29:32.035066 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:32.034889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:42293)
0925 02:29:32.035004 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:32.035029 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":72,"threads_started":1}
I20260925 02:29:32.035398 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:32.035319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:39149)
0925 02:29:32.035350 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:32.035374 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260925 02:29:32.092209 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:32.092029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:44117)
0925 02:29:32.092141 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:32.092169 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":51,"threads_started":1}
I20260925 02:29:33.035825 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:33.035570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:33183)
0925 02:29:33.035718 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:33.035758 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":105,"threads_started":1}
I20260925 02:29:33.035984 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:33.035781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:54547)
0925 02:29:33.035903 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:33.035934 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260925 02:29:33.092762 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:33.092588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:49741)
0925 02:29:33.092692 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:33.092721 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260925 02:29:34.036480 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:34.036246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:59999)
0925 02:29:34.036367 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:34.036405 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
I20260925 02:29:34.036607 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:34.036359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:38933)
0925 02:29:34.036523 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:34.036559 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":70,"threads_started":1}
I20260925 02:29:34.093345 27716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:34.093157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:57047)
0925 02:29:34.093269 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:34.093300 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260925 02:29:35.037111 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:35.036922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:39557)
0925 02:29:35.037049 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:35.037078 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":44,"threads_started":1}
I20260925 02:29:35.037111 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:35.036905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:53297)
0925 02:29:35.037023 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:35.037056 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260925 02:29:35.093946 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:35.093743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:39337)
0925 02:29:35.093868 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:35.093902 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260925 02:29:36.037652 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:36.037464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:48613)
0925 02:29:36.037576 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:36.037602 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":69,"threads_started":1}
I20260925 02:29:36.037750 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:36.037550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:49943)
0925 02:29:36.037664 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:36.037690 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
I20260925 02:29:36.094509 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:36.094337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:40107)
0925 02:29:36.094435 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:36.094466 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
I20260925 02:29:37.038251 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:37.037967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:39029)
0925 02:29:37.038181 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:37.038207 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":78,"threads_started":1}
I20260925 02:29:37.038254 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:37.038082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:39231)
0925 02:29:37.038175 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:37.038209 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":40,"threads_started":1}
I20260925 02:29:37.095141 27725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:37.094918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:58033)
0925 02:29:37.095053 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:37.095085 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260925 02:29:38.038790 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:38.038603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:46125)
0925 02:29:38.038724 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:38.038749 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260925 02:29:38.038959 27727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:38.038768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:43985)
0925 02:29:38.038871 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:38.038905 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":62,"threads_started":1}
I20260925 02:29:38.095901 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:38.095711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:57181)
0925 02:29:38.095825 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:38.095858 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":53,"threads_started":1}
I20260925 02:29:39.039592 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:39.039384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:53277)
0925 02:29:39.039503 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:39.039539 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":52,"threads_started":1}
I20260925 02:29:39.039651 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:39.039468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:46521)
0925 02:29:39.039583 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:39.039612 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":55,"threads_started":1}
I20260925 02:29:39.096536 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:39.096304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:55879)
0925 02:29:39.096419 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:39.096492 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":51,"threads_started":1}
I20260925 02:29:40.040300 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:40.040109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:46611)
0925 02:29:40.040225 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:40.040256 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":54,"threads_started":1}
I20260925 02:29:40.040320 27732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:40.040109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:51317)
0925 02:29:40.040254 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:40.040281 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":68,"threads_started":1}
I20260925 02:29:40.097105 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:40.096932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:38667)
0925 02:29:40.097034 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:40.097063 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260925 02:29:41.040861 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:41.040684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.194:45299)
0925 02:29:41.040796 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:41.040817 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260925 02:29:41.040875 27735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:41.040683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.195:46575)
0925 02:29:41.040797 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:41.040827 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
I20260925 02:29:41.097735 27737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:41.097531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:45041 (local address 127.11.220.193:37731)
0925 02:29:41.097653 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:41.097689 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:45041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":55,"threads_started":1}
I20260925 02:29:41.405164 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 27235
I20260925 02:29:41.409695 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 27399
I20260925 02:29:41.414024 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 27539
2026-09-25T02:29:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18015 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:29:41 dist-test-slave-dn4n krb5kdc[27741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:29:41 dist-test-slave-dn4n krb5kdc[27741](info): set up 2 sockets
Sep 25 02:29:41 dist-test-slave-dn4n krb5kdc[27741](info): commencing operation
krb5kdc: starting...
W20260925 02:29:42.353130 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:29:42 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:29:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:29:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:29:42.452841 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43577
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:43577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:42.524613 27754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:42.524765 27754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:42.524782 27754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:42.525681 27754 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 02:29:42.525722 27754 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 02:29:42.525734 27754 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 02:29:42.525745 27754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:29:42.525801 27754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:42.525822 27754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:42.525856 27754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:29:42.525875 27754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:29:42.527215 27754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:43577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:42.527375 27754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:42.527504 27754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:42.531508 27760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:42.531517 27759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:42.531577 27754 server_base.cc:1034] running on GCE node
W20260925 02:29:42.531517 27762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:42.531916 27754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:42.532133 27754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:42.533273 27754 hybrid_clock.cc:648] HybridClock initialized: now 1790303382533273 us; error 33 us; skew 500 ppm
Sep 25 02:29:42 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:42.535292 27754 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:29:42.535691 27754 webserver.cc:460] Webserver started at http://127.11.220.254:40245/ using document root <none> and password file <none>
I20260925 02:29:42.535876 27754 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:42.535944 27754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:42.536041 27754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:42.536973 27754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "67b5ba0c9c644455895bf173fe297cb1"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "108cac9179fc7a61cc697e2750ffdb56"
server_key_iv: "376b58385d556302965fdd5d4e4de1ed"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.537237 27754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "67b5ba0c9c644455895bf173fe297cb1"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "108cac9179fc7a61cc697e2750ffdb56"
server_key_iv: "376b58385d556302965fdd5d4e4de1ed"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.538324 27754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:29:42.538869 27769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:42.539012 27754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:42.539095 27754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "67b5ba0c9c644455895bf173fe297cb1"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "108cac9179fc7a61cc697e2750ffdb56"
server_key_iv: "376b58385d556302965fdd5d4e4de1ed"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.539160 27754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:42.548156 27754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:42.549655 27754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:42.549789 27754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:42.553162 27754 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:43577
I20260925 02:29:42.553188 27821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:43577 every 8 connection(s)
I20260925 02:29:42.553447 27754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:29:42.554100 27822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:42.555860 27822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Bootstrap starting.
I20260925 02:29:42.556237 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 27754
I20260925 02:29:42.556344 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:29:42.556360 27822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:42.556614 12147 external_mini_cluster.cc:1428] Setting key 3aa686bb53d6504be643540d7ad5f17c
I20260925 02:29:42.556672 27822 log.cc:824] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:42.557446 27822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: No bootstrap required, opened a new log
I20260925 02:29:42.559526 27822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:29:42.559669 27822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:42.559700 27822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67b5ba0c9c644455895bf173fe297cb1, State: Initialized, Role: FOLLOWER
I20260925 02:29:42.559808 27822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:29:42.559867 27822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:29:42.559891 27822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:29:42.559929 27822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:42.560582 27822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:29:42.560665 27822 leader_election.cc:302] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67b5ba0c9c644455895bf173fe297cb1; no voters:
I20260925 02:29:42.560909 27822 leader_election.cc:288] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:29:42.560932 27827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:29:42.561189 27822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:29:42.561311 27827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 1 LEADER]: Becoming Leader. State: Replica: 67b5ba0c9c644455895bf173fe297cb1, State: Running, Role: LEADER
I20260925 02:29:42.561439 27827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:29:42.561770 27827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67b5ba0c9c644455895bf173fe297cb1. Latest consensus state: current_term: 1 leader_uuid: "67b5ba0c9c644455895bf173fe297cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } } }
I20260925 02:29:42.561777 27828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67b5ba0c9c644455895bf173fe297cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } } }
I20260925 02:29:42.561941 27828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:42.562124 27827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:29:42.562434 27836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 25 02:29:42 dist-test-slave-dn4n krb5kdc[27741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:42.562938 27836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:29:42.564232 27836 catalog_manager.cc:1347] Generated new cluster ID: 3e56a4535ae34a06a6b26b6cbf22e161
I20260925 02:29:42.564280 27836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:29:42.566630 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:42.557057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41936 (local address 127.11.220.254:43577)
0925 02:29:42.557235 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:42.557241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:42.557263 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:29:42.557650 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:42.557655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:42.557695 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:42.557791 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:42.558486 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:42.559213 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:42.560221 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:42.560679 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:42.560786 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:42.563301 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:42.563304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:42.563304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:42.563314 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:42.565157 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:42.565499 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:42.565501 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:42.565502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:42.565594 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:42.565784 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:42.565785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:42.565785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:42.566050 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:42.566222 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:42.566317 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 02:29:42.566436 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260925 02:29:42.585072 27836 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:29:42.585469 27836 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:29:42.588699 27836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Generated new TSK 0
I20260925 02:29:42.588847 27836 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:29:42.616669 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43577
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:42.688921 27848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:42.689078 27848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:42.689098 27848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:42.690111 27848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:42.690145 27848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:42.690168 27848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:29:42.691381 27848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:42.691541 27848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:42.691696 27848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:42.696074 27854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:42.696171 27853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:42.696107 27857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:42.696257 27848 server_base.cc:1034] running on GCE node
I20260925 02:29:42.696530 27848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:42.696807 27848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:42.697942 27848 hybrid_clock.cc:648] HybridClock initialized: now 1790303382697932 us; error 25 us; skew 500 ppm
Sep 25 02:29:42 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:42.700371 27848 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:29:42.700763 27848 webserver.cc:460] Webserver started at http://127.11.220.193:38785/ using document root <none> and password file <none>
I20260925 02:29:42.701014 27848 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:42.701076 27848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:42.701215 27848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:42.702111 27848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "77e43809508c455581624aa26756ff17"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "01a157f7725873bc2538bb0461579795"
server_key_iv: "11c4bad8854053304fae7767846570f7"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.702411 27848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "77e43809508c455581624aa26756ff17"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "01a157f7725873bc2538bb0461579795"
server_key_iv: "11c4bad8854053304fae7767846570f7"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.703547 27848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:29:42.704175 27864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:42.704319 27848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:42.704401 27848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "77e43809508c455581624aa26756ff17"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "01a157f7725873bc2538bb0461579795"
server_key_iv: "11c4bad8854053304fae7767846570f7"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.704494 27848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:42.733441 27848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:42.734997 27848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:42.735137 27848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:42.735325 27848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:42.735586 27848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:42.735639 27848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:42.735671 27848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:42.735697 27848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:42.741149 27848 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:44041
I20260925 02:29:42.741205 27978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:44041 every 8 connection(s)
I20260925 02:29:42.741499 27848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
Sep 25 02:29:42 dist-test-slave-dn4n krb5kdc[27741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:42.749346 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:42.742054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:50197 (local address 127.11.220.254:43577)
0925 02:29:42.742091 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:42.742093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:42.742273 (+ 180us) server_negotiation.cc:407] Connection header received
0925 02:29:42.742771 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:42.742772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:42.742783 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:42.742832 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:42.743713 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:42.744217 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:42.745208 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:42.745400 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:42.745467 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:42.747253 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:42.747255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:42.747255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:42.747261 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:42.748302 (+ 1041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:42.748670 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:42.748670 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:42.748671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:42.748712 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:42.748891 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:42.748892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:42.748892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:42.748990 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:42.749031 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:42.749231 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 02:29:42.749287 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:29:42.749483 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:42.742034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:50197)
0925 02:29:42.742172 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:42.742201 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:42.742678 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:42.742890 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:42.742893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:42.743450 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:42.743615 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:42.743619 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:42.744317 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:42.744318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:42.745106 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:42.745108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:42.745474 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:42.745475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:42.745592 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:42.745945 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:42.745952 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:42.747167 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:42.748365 (+ 1198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:42.748371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:42.748373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:42.748616 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:42.748763 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:42.748764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:42.748764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:42.748849 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:42.749037 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:42.749041 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:42.749139 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 02:29:42.749217 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 02:29:42.749318 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
I20260925 02:29:42.749951 27979 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43577
I20260925 02:29:42.750027 27979 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:42.750145 27979 heartbeater.cc:511] Master 127.11.220.254:43577 requested a full tablet report, sending...
I20260925 02:29:42.750582 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 27848
I20260925 02:29:42.750665 27786 ts_manager.cc:194] Registered new tserver with Master: 77e43809508c455581624aa26756ff17 (127.11.220.193:44041)
I20260925 02:29:42.750701 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:29:42.750918 12147 external_mini_cluster.cc:1428] Setting key 2b8b7ddd587259960f12912e4b7dbdbf
I20260925 02:29:42.751367 27786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:50197
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:29:42.775218 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43577
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:42.848253 27984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:42.848410 27984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:42.848430 27984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:42.849453 27984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:42.849489 27984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:42.849512 27984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:29:42.850760 27984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:42.850906 27984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:42.851043 27984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:42.855386 27989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:42.855394 27992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:42.855436 27984 server_base.cc:1034] running on GCE node
W20260925 02:29:42.855465 27990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:42.855734 27984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:42.855938 27984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:42.857086 27984 hybrid_clock.cc:648] HybridClock initialized: now 1790303382857066 us; error 37 us; skew 500 ppm
Sep 25 02:29:42 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:42.859331 27984 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:29:42.859736 27984 webserver.cc:460] Webserver started at http://127.11.220.194:37525/ using document root <none> and password file <none>
I20260925 02:29:42.859930 27984 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:42.860019 27984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:42.860172 27984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:42.861250 27984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "6b4efb5d48d94633bac21729213578d7"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "8fb07fa5019f8b605cccb0f1d1af9392"
server_key_iv: "7b133f70183cfbef06e5ef73f4353462"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.861593 27984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "6b4efb5d48d94633bac21729213578d7"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "8fb07fa5019f8b605cccb0f1d1af9392"
server_key_iv: "7b133f70183cfbef06e5ef73f4353462"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.862854 27984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:29:42.863543 27999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:42.863708 27984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:29:42.863797 27984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "6b4efb5d48d94633bac21729213578d7"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "8fb07fa5019f8b605cccb0f1d1af9392"
server_key_iv: "7b133f70183cfbef06e5ef73f4353462"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.863875 27984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:42.871429 27984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:42.873003 27984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:42.873143 27984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:42.873325 27984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:42.873579 27984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:42.873624 27984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:42.873652 27984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:42.873675 27984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:42.879000 27984 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:45495
I20260925 02:29:42.879045 28113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:45495 every 8 connection(s)
I20260925 02:29:42.879287 27984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:29:42 dist-test-slave-dn4n krb5kdc[27741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:42.886943 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:42.879852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:48921 (local address 127.11.220.254:43577)
0925 02:29:42.879892 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:42.879894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:42.880098 (+ 204us) server_negotiation.cc:407] Connection header received
0925 02:29:42.880545 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:42.880547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:42.880560 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:42.880605 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:42.881399 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:42.881841 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:42.882747 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:42.882933 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:42.883001 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:42.884751 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:42.884752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:42.884752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:42.884757 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:42.885785 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:42.886208 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:42.886209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:42.886210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:42.886262 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:42.886474 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:42.886475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:42.886475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:42.886580 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:42.886628 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:42.886828 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 02:29:42.886886 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:29:42.887055 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:42.879827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:48921)
0925 02:29:42.879982 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:42.880025 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:42.880423 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:42.880659 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:42.880662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:42.881144 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:42.881298 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:42.881302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:42.881932 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:42.881933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:42.882663 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:42.882665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:42.883016 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:42.883017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:42.883124 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:42.883540 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:42.883547 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:42.884678 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:42.885851 (+ 1173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:42.885857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:42.885859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:42.886041 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:42.886330 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:42.886330 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:42.886331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:42.886417 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:42.886631 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:42.886635 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:42.886728 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 02:29:42.886801 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:29:42.886900 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260925 02:29:42.887576 28114 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43577
I20260925 02:29:42.887692 28114 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:42.887850 28114 heartbeater.cc:511] Master 127.11.220.254:43577 requested a full tablet report, sending...
I20260925 02:29:42.888242 27786 ts_manager.cc:194] Registered new tserver with Master: 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495)
I20260925 02:29:42.888689 27786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:48921
I20260925 02:29:42.889134 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 27984
I20260925 02:29:42.889211 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:29:42.889437 12147 external_mini_cluster.cc:1428] Setting key a59a558f2bb5a14a76e69adbfb85b9b8
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:29:42.911038 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43577
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:29:42.983788 28119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:29:42.983940 28119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:29:42.983960 28119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:29:42.984973 28119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:29:42.985008 28119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:29:42.985029 28119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:29:42.986227 28119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:29:42.986385 28119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:29:42.986524 28119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:29:42.990877 28124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:42.990974 28119 server_base.cc:1034] running on GCE node
W20260925 02:29:42.990893 28127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:29:42.990891 28125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:29:42.991282 28119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:29:42.991446 28119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:29:42.992623 28119 hybrid_clock.cc:648] HybridClock initialized: now 1790303382992603 us; error 39 us; skew 500 ppm
Sep 25 02:29:42 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:29:42.994709 28119 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:29:42.995065 28119 webserver.cc:460] Webserver started at http://127.11.220.195:37563/ using document root <none> and password file <none>
I20260925 02:29:42.995273 28119 fs_manager.cc:359] Metadata directory not provided
I20260925 02:29:42.995333 28119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:29:42.995469 28119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:29:42.996380 28119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "7cde30baee774406bfed9ec5cde16213"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "4b4f530182849b16f4ba90b5295f7dd0"
server_key_iv: "0b1cb24f385fdbf8249c2405a6cf74d2"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.996784 28119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "7cde30baee774406bfed9ec5cde16213"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "4b4f530182849b16f4ba90b5295f7dd0"
server_key_iv: "0b1cb24f385fdbf8249c2405a6cf74d2"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.997953 28119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:29:42.998557 28134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:42.998724 28119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:29:42.998807 28119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "7cde30baee774406bfed9ec5cde16213"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "4b4f530182849b16f4ba90b5295f7dd0"
server_key_iv: "0b1cb24f385fdbf8249c2405a6cf74d2"
server_key_version: "encryptionkey@0"
I20260925 02:29:42.998884 28119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:29:43.008292 28119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:29:43.009804 28119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:29:43.009940 28119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:29:43.010125 28119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:29:43.010381 28119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:29:43.010428 28119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:43.010464 28119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:29:43.010489 28119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:29:43.015457 28119 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:40099
I20260925 02:29:43.015516 28249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:40099 every 8 connection(s)
I20260925 02:29:43.015807 28119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:29:43 dist-test-slave-dn4n krb5kdc[27741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303382, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:29:43.023892 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.016472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:58351 (local address 127.11.220.254:43577)
0925 02:29:43.016536 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:43.016538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:43.016729 (+ 191us) server_negotiation.cc:407] Connection header received
0925 02:29:43.017213 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:43.017214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:43.017227 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:43.017309 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:43.018091 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.018542 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.019421 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.019606 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.019673 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.021616 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:43.021617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:43.021617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:43.021625 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:43.022678 (+ 1053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:43.023034 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:43.023036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:43.023036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:43.023090 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:43.023326 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:43.023327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:43.023327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:43.023425 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:43.023491 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:43.023730 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 02:29:43.023812 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:29:43.024021 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.016413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:58351)
0925 02:29:43.016612 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:43.016654 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:43.017101 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:43.017364 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:43.017368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:43.017845 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:29:43.017994 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.017997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.018640 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.018641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.019330 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.019332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.019680 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.019681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.019817 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.020275 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:29:43.020283 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:29:43.021529 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:29:43.022753 (+ 1224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:43.022759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:43.022762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:43.022952 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:43.023154 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:29:43.023155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:29:43.023155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:29:43.023254 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:29:43.023492 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:29:43.023499 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:29:43.023613 (+ 114us) client_negotiation.cc:769] Sending connection context
0925 02:29:43.023707 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 02:29:43.023834 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":118,"threads_started":1}
I20260925 02:29:43.024497 28250 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43577
I20260925 02:29:43.024593 28250 heartbeater.cc:464] Registering TS with master...
I20260925 02:29:43.024665 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 28119
I20260925 02:29:43.024772 28250 heartbeater.cc:511] Master 127.11.220.254:43577 requested a full tablet report, sending...
I20260925 02:29:43.024780 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:29:43.024956 12147 external_mini_cluster.cc:1428] Setting key 6165792ba8aeb13cde90ba9f037557fa
I20260925 02:29:43.025125 27786 ts_manager.cc:194] Registered new tserver with Master: 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099)
I20260925 02:29:43.025553 27786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:58351
I20260925 02:29:43.025751 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:29:43.034569 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.027112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41944 (local address 127.11.220.254:43577)
0925 02:29:43.027176 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:43.027178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:43.027283 (+ 105us) server_negotiation.cc:407] Connection header received
0925 02:29:43.027365 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:43.027366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:43.027375 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:43.027435 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:29:43.028122 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.028881 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.029948 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.030258 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.030314 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.031818 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:29:43.031834 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:29:43.031835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:29:43.031842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:29:43.033398 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:43.033804 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:43.033805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:43.033805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:43.033893 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:29:43.034065 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:29:43.034066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:29:43.034067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:29:43.034203 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:29:43.034304 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:43.034419 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 02:29:43.034487 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:29:43.036424 27786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:29:43.037873 27786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:29:43.043460 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.039725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40099 (local address 127.0.0.1:46264)
0925 02:29:43.039824 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:43.039858 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:43.039916 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:43.040202 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:43.040206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:43.040208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:43.040260 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.040261 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.041302 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.041303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.042326 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.042329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.043178 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.043178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.043256 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.043270 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 02:29:43.043348 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 02:29:43.043397 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":58,"threads_started":1}
I20260925 02:29:43.043574 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.039738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44041 (local address 127.0.0.1:35988)
0925 02:29:43.040010 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:43.040022 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:43.040081 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:43.040466 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:43.040467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:43.040469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:43.040517 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.040519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.041777 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.041778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.042713 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.042715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.043364 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.043365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.043422 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.043425 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:29:43.043470 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 02:29:43.043506 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":40,"threads_started":1}
I20260925 02:29:43.043788 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.039862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46264 (local address 127.11.220.195:40099)
0925 02:29:43.040015 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:43.040019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:43.040026 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:29:43.040053 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:43.040055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:43.040071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:43.040136 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:43.040317 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.041206 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.042464 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.043097 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.043163 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.043208 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:43.043664 (+ 456us) server_negotiation.cc:299] Negotiation successful
0925 02:29:43.043715 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260925 02:29:43.043865 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.039842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.11.220.193:44041)
0925 02:29:43.040288 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:43.040292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:43.040302 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:29:43.040339 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:43.040340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:43.040354 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:43.040593 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:43.040680 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.041678 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.042789 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.043278 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.043357 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.043401 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:43.043480 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 02:29:43.043789 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":70,"threads_started":1}
I20260925 02:29:43.044262 28183 tablet_service.cc:1467] Processing CreateTablet for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4 (DEFAULT_TABLE table=test-table [id=f658074cb6414bf48ef8ecee0f1c23eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:43.044600 28183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:43.044448 27913 tablet_service.cc:1467] Processing CreateTablet for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4 (DEFAULT_TABLE table=test-table [id=f658074cb6414bf48ef8ecee0f1c23eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:43.044786 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.039907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:45495 (local address 127.0.0.1:37514)
0925 02:29:43.040126 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:43.040135 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:43.040169 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:43.040485 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:43.040485 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:43.040486 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:43.040517 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.040519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.042791 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.042791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.043732 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.043735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.044574 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.044576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.044637 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.044642 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:29:43.044693 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:29:43.044732 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":74,"threads_started":1}
I20260925 02:29:43.044885 27913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:43.044993 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.040129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.11.220.194:45495)
0925 02:29:43.040290 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:43.040293 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:43.040302 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:29:43.040338 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:43.040340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:43.040376 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:43.040440 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:43.040629 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.041673 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.043794 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.044497 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.044568 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.044615 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:43.044871 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 02:29:43.044919 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260925 02:29:43.045591 28047 tablet_service.cc:1467] Processing CreateTablet for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4 (DEFAULT_TABLE table=test-table [id=f658074cb6414bf48ef8ecee0f1c23eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:29:43.045894 28047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:29:43.047204 28264 tablet_bootstrap.cc:492] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Bootstrap starting.
I20260925 02:29:43.047834 28264 tablet_bootstrap.cc:654] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:43.048069 28265 tablet_bootstrap.cc:492] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: Bootstrap starting.
I20260925 02:29:43.048403 28264 log.cc:824] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:43.048718 28265 tablet_bootstrap.cc:654] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:43.049218 28265 log.cc:824] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:43.049291 28264 tablet_bootstrap.cc:492] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: No bootstrap required, opened a new log
I20260925 02:29:43.049342 28264 ts_tablet_manager.cc:1397] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:29:43.049899 28265 tablet_bootstrap.cc:492] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: No bootstrap required, opened a new log
I20260925 02:29:43.050024 28265 ts_tablet_manager.cc:1397] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:29:43.051151 28264 raft_consensus.cc:348] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:29:43.051324 28264 raft_consensus.cc:374] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:43.051359 28264 raft_consensus.cc:729] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e43809508c455581624aa26756ff17, State: Initialized, Role: FOLLOWER
I20260925 02:29:43.051463 28264 consensus_queue.cc:260] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:29:43.051728 28264 ts_tablet_manager.cc:1428] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:29:43.051751 27979 heartbeater.cc:503] Master 127.11.220.254:43577 was elected leader, sending a full tablet report...
I20260925 02:29:43.051883 28267 tablet_bootstrap.cc:492] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: Bootstrap starting.
I20260925 02:29:43.051846 28265 raft_consensus.cc:348] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:29:43.052062 28265 raft_consensus.cc:374] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:43.052150 28265 raft_consensus.cc:729] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cde30baee774406bfed9ec5cde16213, State: Initialized, Role: FOLLOWER
I20260925 02:29:43.052306 28265 consensus_queue.cc:260] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:29:43.052490 28267 tablet_bootstrap.cc:654] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: Neither blocks nor log segments found. Creating new log.
I20260925 02:29:43.052640 28265 ts_tablet_manager.cc:1428] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 02:29:43.052667 28250 heartbeater.cc:503] Master 127.11.220.254:43577 was elected leader, sending a full tablet report...
I20260925 02:29:43.053049 28267 log.cc:824] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: Log is configured to *not* fsync() on all Append() calls
I20260925 02:29:43.053838 28267 tablet_bootstrap.cc:492] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: No bootstrap required, opened a new log
I20260925 02:29:43.053886 28267 ts_tablet_manager.cc:1397] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:29:43.055045 28267 raft_consensus.cc:348] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:29:43.055174 28267 raft_consensus.cc:374] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:29:43.055217 28267 raft_consensus.cc:729] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4efb5d48d94633bac21729213578d7, State: Initialized, Role: FOLLOWER
I20260925 02:29:43.055311 28267 consensus_queue.cc:260] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:29:43.055544 28267 ts_tablet_manager.cc:1428] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:29:43.055589 28114 heartbeater.cc:503] Master 127.11.220.254:43577 was elected leader, sending a full tablet report...
I20260925 02:29:43.062567 28272 raft_consensus.cc:482] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:29:43.062693 28272 raft_consensus.cc:504] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:29:43.063014 28272 leader_election.cc:288] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099), 77e43809508c455581624aa26756ff17 (127.11.220.193:44041)
I20260925 02:29:43.065543 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.063235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:36935 (local address 127.11.220.193:44041)
0925 02:29:43.063274 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:43.063275 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:43.063318 (+ 43us) server_negotiation.cc:407] Connection header received
0925 02:29:43.063339 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:43.063339 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:43.063348 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:43.063392 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:43.063519 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.063944 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.064809 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.065210 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.065292 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.065299 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:43.065440 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 02:29:43.065484 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:29:43.065541 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.063176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:46311 (local address 127.11.220.195:40099)
0925 02:29:43.063222 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:43.063223 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:43.063363 (+ 140us) server_negotiation.cc:407] Connection header received
0925 02:29:43.063408 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:43.063408 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:43.063416 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:43.063469 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:29:43.063607 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.063954 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.064886 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.065212 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.065292 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.065299 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:43.065441 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 02:29:43.065485 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 02:29:43.065553 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.063138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44041 (local address 127.11.220.194:36935)
0925 02:29:43.063216 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:43.063227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:43.063272 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:43.063401 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:43.063402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:43.063403 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:43.063459 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.063461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.064011 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.064012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.064738 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.064740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.065289 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.065290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.065368 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.065371 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:29:43.065435 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:29:43.065487 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":1}
I20260925 02:29:43.065562 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.063138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40099 (local address 127.11.220.194:46311)
0925 02:29:43.063292 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:43.063311 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 02:29:43.063361 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:29:43.063477 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:29:43.063477 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:29:43.063478 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:29:43.063551 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.063552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.064037 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.064038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.064813 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:29:43.064814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.065292 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.065293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:29:43.065368 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.065371 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:29:43.065435 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:29:43.065483 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":10,"thread_start_us":63,"threads_started":1}
I20260925 02:29:43.065867 28203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8ec17a172c4c60aeb1a7456dfc4ba4" candidate_uuid: "6b4efb5d48d94633bac21729213578d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cde30baee774406bfed9ec5cde16213" is_pre_election: true
I20260925 02:29:43.065963 28203 raft_consensus.cc:2454] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4efb5d48d94633bac21729213578d7 in term 0.
I20260925 02:29:43.065989 27933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8ec17a172c4c60aeb1a7456dfc4ba4" candidate_uuid: "6b4efb5d48d94633bac21729213578d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e43809508c455581624aa26756ff17" is_pre_election: true
I20260925 02:29:43.066107 27933 raft_consensus.cc:2454] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b4efb5d48d94633bac21729213578d7 in term 0.
I20260925 02:29:43.066198 28001 leader_election.cc:302] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4efb5d48d94633bac21729213578d7, 7cde30baee774406bfed9ec5cde16213; no voters:
I20260925 02:29:43.066335 28272 raft_consensus.cc:2781] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:29:43.066399 28272 raft_consensus.cc:482] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:29:43.066442 28272 raft_consensus.cc:3037] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:43.067032 28272 raft_consensus.cc:504] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:29:43.067158 28272 leader_election.cc:288] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [CANDIDATE]: Term 1 election: Requested vote from peers 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099), 77e43809508c455581624aa26756ff17 (127.11.220.193:44041)
I20260925 02:29:43.067310 28203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8ec17a172c4c60aeb1a7456dfc4ba4" candidate_uuid: "6b4efb5d48d94633bac21729213578d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cde30baee774406bfed9ec5cde16213"
I20260925 02:29:43.067313 27933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea8ec17a172c4c60aeb1a7456dfc4ba4" candidate_uuid: "6b4efb5d48d94633bac21729213578d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e43809508c455581624aa26756ff17"
I20260925 02:29:43.067373 28203 raft_consensus.cc:3037] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:43.067394 27933 raft_consensus.cc:3037] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:29:43.067862 28203 raft_consensus.cc:2454] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4efb5d48d94633bac21729213578d7 in term 1.
I20260925 02:29:43.068003 28001 leader_election.cc:302] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4efb5d48d94633bac21729213578d7, 7cde30baee774406bfed9ec5cde16213; no voters:
I20260925 02:29:43.068032 27933 raft_consensus.cc:2454] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b4efb5d48d94633bac21729213578d7 in term 1.
I20260925 02:29:43.068086 28272 raft_consensus.cc:2781] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:29:43.068222 28272 raft_consensus.cc:686] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 1 LEADER]: Becoming Leader. State: Replica: 6b4efb5d48d94633bac21729213578d7, State: Running, Role: LEADER
I20260925 02:29:43.068305 28272 consensus_queue.cc:237] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:29:43.069092 27785 catalog_manager.cc:5495] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b4efb5d48d94633bac21729213578d7 (127.11.220.194). New cstate: current_term: 1 leader_uuid: "6b4efb5d48d94633bac21729213578d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } health_report { overall_health: HEALTHY } } }
I20260925 02:29:43.080176 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.078462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.11.220.194:45495)
0925 02:29:43.078494 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:29:43.078496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:29:43.078503 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:29:43.078547 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:29:43.078547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:29:43.078559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:29:43.078610 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:29:43.078689 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.079050 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.079655 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:29:43.079805 (+ 150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:29:43.079847 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:29:43.079958 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:29:43.080037 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:29:43.080085 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:29:43.080122 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 02:29:43.080136 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:29:43.081667 27933 raft_consensus.cc:1261] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Refusing update from remote peer 6b4efb5d48d94633bac21729213578d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:29:43.081697 28203 raft_consensus.cc:1261] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213 [term 1 FOLLOWER]: Refusing update from remote peer 6b4efb5d48d94633bac21729213578d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:29:43.081945 28272 consensus_queue.cc:1035] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:29:43.082055 28274 consensus_queue.cc:1035] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:29:43.084260 28277 mvcc.cc:205] Tried to move back new op lower bound from 7333082657100234752 to 7333082657048555520. Current Snapshot: MvccSnapshot[applied={T|T < 7333082657100234752}]
I20260925 02:29:43.084877 28278 mvcc.cc:205] Tried to move back new op lower bound from 7333082657100234752 to 7333082657048555520. Current Snapshot: MvccSnapshot[applied={T|T < 7333082657100234752}]
I20260925 02:29:43.085043 28280 mvcc.cc:205] Tried to move back new op lower bound from 7333082657100234752 to 7333082657048555520. Current Snapshot: MvccSnapshot[applied={T|T < 7333082657100234752}]
I20260925 02:29:43.086205 27785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:29:43.086274 27785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:29:43.086876 27785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Sending DeleteTablet for 3 replicas of tablet ea8ec17a172c4c60aeb1a7456dfc4ba4
I20260925 02:29:43.087157 27913 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:43 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:35988
I20260925 02:29:43.087213 28183 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:43 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:46264
I20260925 02:29:43.087363 28288 tablet_replica.cc:330] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: stopping tablet replica
I20260925 02:29:43.087369 28047 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:43 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:37514
I20260925 02:29:43.087457 28288 raft_consensus.cc:2229] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:29:43.087539 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 27848
I20260925 02:29:43.087541 28290 tablet_replica.cc:330] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: stopping tablet replica
I20260925 02:29:43.087599 28288 raft_consensus.cc:2258] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:29:43.087641 28290 raft_consensus.cc:2229] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:29:43.087723 28290 raft_consensus.cc:2258] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:29:43.089469 28289 tablet_replica.cc:330] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: stopping tablet replica
I20260925 02:29:43.089584 28289 raft_consensus.cc:2229] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:29:43.089707 28289 raft_consensus.cc:2258] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:29:43.089835 28290 ts_tablet_manager.cc:1905] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:29:43.090111 28289 ts_tablet_manager.cc:1905] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:29:43.091598 28290 ts_tablet_manager.cc:1918] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:29:43.091748 28290 log.cc:1196] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/ea8ec17a172c4c60aeb1a7456dfc4ba4
I20260925 02:29:43.091773 28289 ts_tablet_manager.cc:1918] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:29:43.091881 28289 log.cc:1196] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/ea8ec17a172c4c60aeb1a7456dfc4ba4
I20260925 02:29:43.092134 28290 ts_tablet_manager.cc:1939] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 6b4efb5d48d94633bac21729213578d7: Deleting consensus metadata
I20260925 02:29:43.092180 28289 ts_tablet_manager.cc:1939] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 7cde30baee774406bfed9ec5cde16213: Deleting consensus metadata
I20260925 02:29:43.092674 27771 catalog_manager.cc:4841] TS 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099): tablet ea8ec17a172c4c60aeb1a7456dfc4ba4 (table test-table [id=f658074cb6414bf48ef8ecee0f1c23eb]) successfully deleted
I20260925 02:29:43.093050 27771 catalog_manager.cc:4841] TS 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495): tablet ea8ec17a172c4c60aeb1a7456dfc4ba4 (table test-table [id=f658074cb6414bf48ef8ecee0f1c23eb]) successfully deleted
W20260925 02:29:43.093829 27772 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:44041 (error 108)
I20260925 02:29:43.094002 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 27984
I20260925 02:29:43.096779 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:29:43.096679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44041 (local address 127.0.0.1:35996)
0925 02:29:43.096726 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 02:29:43.096758 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:44041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 02:29:43.096911 27772 catalog_manager.cc:4584] TS 77e43809508c455581624aa26756ff17 (127.11.220.193:44041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4: Network error: Client connection negotiation failed: client connection to 127.11.220.193:44041: connect: Connection refused (error 111)
I20260925 02:29:43.098958 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 28119
I20260925 02:29:43.103330 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 27754
I20260925 02:30:03.108606 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43577
--webserver_interface=127.11.220.254
--webserver_port=40245
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:43577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:03.187609 28292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:03.187775 28292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:03.187795 28292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:03.188997 28292 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 02:30:03.189033 28292 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 02:30:03.189047 28292 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 02:30:03.189059 28292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:03.189102 28292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:03.189121 28292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:03.189146 28292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:03.189174 28292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:03.190723 28292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:43577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=40245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:03.190943 28292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:03.191149 28292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:03.198843 28299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:03.198864 28298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:03.198869 28301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:03.199086 28292 server_base.cc:1034] running on GCE node
I20260925 02:30:03.199283 28292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:03.199548 28292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:03.200667 28292 hybrid_clock.cc:648] HybridClock initialized: now 1790303403200654 us; error 26 us; skew 500 ppm
Sep 25 02:30:03 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:03.203070 28292 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:30:03.203464 28292 webserver.cc:460] Webserver started at http://127.11.220.254:40245/ using document root <none> and password file <none>
I20260925 02:30:03.203636 28292 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:03.203679 28292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:03.204839 28292 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:03.205333 28308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:03.205493 28292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:03.205569 28292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "67b5ba0c9c644455895bf173fe297cb1"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "108cac9179fc7a61cc697e2750ffdb56"
server_key_iv: "376b58385d556302965fdd5d4e4de1ed"
server_key_version: "encryptionkey@0"
I20260925 02:30:03.205875 28292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:03.224218 28292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:03.226691 28292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:03.226877 28292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:03.230729 28292 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:43577
I20260925 02:30:03.230755 28360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:43577 every 8 connection(s)
I20260925 02:30:03.231083 28292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:03.232270 28361 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:30:03.232882 28361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Bootstrap starting.
I20260925 02:30:03.233335 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 28292
I20260925 02:30:03.233590 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:44041
--local_ip_for_outbound_sockets=127.11.220.193
--tserver_master_addrs=127.11.220.254:43577
--webserver_port=38785
--webserver_interface=127.11.220.193
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:30:03.241434 28361 log.cc:824] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:03.242986 28361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:30:03.243251 28361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Bootstrap complete.
I20260925 02:30:03.245234 28361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:30:03.245425 28361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67b5ba0c9c644455895bf173fe297cb1, State: Initialized, Role: FOLLOWER
I20260925 02:30:03.245585 28361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:30:03.245652 28361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:03.245705 28361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:03.245747 28361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 1 FOLLOWER]: Advancing to term 2
I20260925 02:30:03.246596 28361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:30:03.246675 28361 leader_election.cc:302] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67b5ba0c9c644455895bf173fe297cb1; no voters:
I20260925 02:30:03.246856 28361 leader_election.cc:288] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 02:30:03.246886 28365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 02:30:03.247028 28365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 2 LEADER]: Becoming Leader. State: Replica: 67b5ba0c9c644455895bf173fe297cb1, State: Running, Role: LEADER
I20260925 02:30:03.247090 28361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:03.247152 28365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:30:03.247419 28365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67b5ba0c9c644455895bf173fe297cb1. Latest consensus state: current_term: 2 leader_uuid: "67b5ba0c9c644455895bf173fe297cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } } }
I20260925 02:30:03.247422 28366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "67b5ba0c9c644455895bf173fe297cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } } }
I20260925 02:30:03.247692 28366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:03.247969 28365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:03.248015 28374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:03.248842 28374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:03.249003 28374 catalog_manager.cc:1259] Loaded cluster ID: 3e56a4535ae34a06a6b26b6cbf22e161
I20260925 02:30:03.249245 28374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:03.250391 28374 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:03.250595 28374 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Loaded TSK: 0
I20260925 02:30:03.260473 28374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Generated new TSK 1
I20260925 02:30:03.260550 28374 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:30:03.310089 28363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:03.310278 28363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:03.310312 28363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:03.311328 28363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:03.311381 28363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:03.311422 28363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:30:03.312770 28363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:44041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=38785
--tserver_master_addrs=127.11.220.254:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:03.312973 28363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:03.313163 28363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:03.317368 28388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:03.317404 28391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:03.317533 28363 server_base.cc:1034] running on GCE node
W20260925 02:30:03.317374 28389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:03.317806 28363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:03.318024 28363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:03.319176 28363 hybrid_clock.cc:648] HybridClock initialized: now 1790303403319164 us; error 29 us; skew 500 ppm
Sep 25 02:30:03 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:03.321666 28363 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:30:03.322150 28363 webserver.cc:460] Webserver started at http://127.11.220.193:38785/ using document root <none> and password file <none>
I20260925 02:30:03.322371 28363 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:03.322443 28363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:03.323949 28363 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:30:03.324764 28398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:03.324934 28363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:03.325016 28363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "77e43809508c455581624aa26756ff17"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "01a157f7725873bc2538bb0461579795"
server_key_iv: "11c4bad8854053304fae7767846570f7"
server_key_version: "encryptionkey@0"
I20260925 02:30:03.325282 28363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:03.334991 28363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:03.336946 28363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:03.337081 28363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:03.337302 28363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:03.337702 28405 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 02:30:03.339484 28363 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 02:30:03.339531 28363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 02:30:03.339574 28363 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 02:30:03.340124 28363 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 02:30:03.340171 28363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 02:30:03.340233 28405 tablet_bootstrap.cc:492] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Bootstrap starting.
I20260925 02:30:03.346578 28363 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:44041
I20260925 02:30:03.346910 28363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:30:03.348112 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 28363
I20260925 02:30:03.348371 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:45495
--local_ip_for_outbound_sockets=127.11.220.194
--tserver_master_addrs=127.11.220.254:43577
--webserver_port=37525
--webserver_interface=127.11.220.194
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:30:03.350739 28405 log.cc:824] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:03.352627 28514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:44041 every 8 connection(s)
I20260925 02:30:03.353152 28405 tablet_bootstrap.cc:492] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:30:03.354290 28405 tablet_bootstrap.cc:492] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Bootstrap complete.
Sep 25 02:30:03 dist-test-slave-dn4n krb5kdc[27741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:03.355531 28405 ts_tablet_manager.cc:1397] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260925 02:30:03.356695 28405 raft_consensus.cc:348] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:30:03.356968 28405 raft_consensus.cc:729] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e43809508c455581624aa26756ff17, State: Initialized, Role: FOLLOWER
I20260925 02:30:03.357082 28405 consensus_queue.cc:260] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } } peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } }
I20260925 02:30:03.357300 28405 ts_tablet_manager.cc:1428] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:30:03.358513 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.347525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:54049)
0925 02:30:03.347684 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:03.347720 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:03.349118 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:03.349714 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:03.349718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:03.349991 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:03.350180 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.350185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.351487 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.351488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.352218 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.352220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.353025 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.353027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.353178 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.353763 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:03.353772 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:03.355206 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:03.356965 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:03.356972 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:03.356974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:03.357232 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:03.357471 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:03.357472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:03.357473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:03.357605 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:03.357967 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:03.357973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:03.358095 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 02:30:03.358188 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 02:30:03.358340 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260925 02:30:03.358541 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.347859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:54049 (local address 127.11.220.254:43577)
0925 02:30:03.348107 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.348116 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.348155 (+ 39us) server_negotiation.cc:407] Connection header received
0925 02:30:03.349358 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.349364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.349418 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.349524 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:03.350290 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.351369 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.352309 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.352908 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.353426 (+ 518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.355303 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:03.355307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:03.355307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:03.355318 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:03.356865 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:03.357307 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:03.357309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:03.357310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:03.357410 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:03.357672 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:03.357673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:03.357673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:03.357904 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:03.357983 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.358240 (+ 257us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.358347 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":70,"threads_started":1}
I20260925 02:30:03.359076 28515 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43577
I20260925 02:30:03.359170 28515 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:03.359280 28515 heartbeater.cc:511] Master 127.11.220.254:43577 requested a full tablet report, sending...
I20260925 02:30:03.359892 28325 ts_manager.cc:194] Registered new tserver with Master: 77e43809508c455581624aa26756ff17 (127.11.220.193:44041)
I20260925 02:30:03.360774 28325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:54049
I20260925 02:30:03.361515 28515 heartbeater.cc:503] Master 127.11.220.254:43577 was elected leader, sending a full tablet report...
Sep 25 02:30:03 dist-test-slave-dn4n krb5kdc[27741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.193@KRBTEST.COM
I20260925 02:30:03.368911 28523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.360435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44041 (local address 127.0.0.1:38116)
0925 02:30:03.360744 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:03.360778 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:03.360854 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:03.361174 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:03.361181 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:03.361520 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:03.361606 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.361608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.362468 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.362469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.363462 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.363465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.363837 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.363838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.363933 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.364313 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:03.364318 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:03.365851 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:03.367684 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:03.367690 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:03.367692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:03.367920 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:03.368190 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:03.368191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:03.368191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:03.368255 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:03.368554 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:03.368560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:03.368607 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 02:30:03.368692 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 02:30:03.368808 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":109,"threads_started":1}
I20260925 02:30:03.368924 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.360477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.11.220.193:44041)
0925 02:30:03.360680 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.360684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.360910 (+ 226us) server_negotiation.cc:407] Connection header received
0925 02:30:03.360948 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.360950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.360980 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.361155 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:03.361686 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.362392 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.363541 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.363772 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.363816 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.365942 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:03.365948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:03.365949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:03.365955 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:03.367587 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:03.368065 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:03.368068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:03.368068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:03.368124 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:03.368315 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:03.368316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:03.368316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:03.368483 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:03.368558 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.368730 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.368837 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":58,"threads_started":1}
I20260925 02:30:03.369598 28448 tablet_service.cc:1514] Processing DeleteTablet for tablet ea8ec17a172c4c60aeb1a7456dfc4ba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:29:43 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:38116
I20260925 02:30:03.370054 28524 tablet_replica.cc:330] stopping tablet replica
I20260925 02:30:03.370172 28524 raft_consensus.cc:2229] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:03.370263 28524 raft_consensus.cc:2258] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:03.370596 28524 ts_tablet_manager.cc:1905] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:03.372974 28524 ts_tablet_manager.cc:1918] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:03.373100 28524 log.cc:1196] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal/wals/ea8ec17a172c4c60aeb1a7456dfc4ba4
I20260925 02:30:03.373377 28524 ts_tablet_manager.cc:1939] T ea8ec17a172c4c60aeb1a7456dfc4ba4 P 77e43809508c455581624aa26756ff17: Deleting consensus metadata
I20260925 02:30:03.373833 28311 catalog_manager.cc:4841] TS 77e43809508c455581624aa26756ff17 (127.11.220.193:44041): tablet ea8ec17a172c4c60aeb1a7456dfc4ba4 (table test-table [id=f658074cb6414bf48ef8ecee0f1c23eb]) successfully deleted
W20260925 02:30:03.434424 28519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:03.434621 28519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:03.434655 28519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:03.435663 28519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:03.435714 28519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:03.435752 28519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:30:03.437058 28519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:45495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=37525
--tserver_master_addrs=127.11.220.254:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:03.437283 28519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:03.437469 28519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:03.441852 28532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:03.441891 28530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:03.441864 28529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:03.442093 28519 server_base.cc:1034] running on GCE node
I20260925 02:30:03.442269 28519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:03.442512 28519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:03.443660 28519 hybrid_clock.cc:648] HybridClock initialized: now 1790303403443649 us; error 35 us; skew 500 ppm
Sep 25 02:30:03 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:03.445781 28519 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:30:03.446182 28519 webserver.cc:460] Webserver started at http://127.11.220.194:37525/ using document root <none> and password file <none>
I20260925 02:30:03.446379 28519 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:03.446450 28519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:03.447700 28519 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:03.448261 28539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:03.448417 28519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:03.448541 28519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "6b4efb5d48d94633bac21729213578d7"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "8fb07fa5019f8b605cccb0f1d1af9392"
server_key_iv: "7b133f70183cfbef06e5ef73f4353462"
server_key_version: "encryptionkey@0"
I20260925 02:30:03.448892 28519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:03.464949 28519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:03.466475 28519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:03.466622 28519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:03.466813 28519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:03.467078 28519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:03.467123 28519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:03.467159 28519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:03.467185 28519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:03.472940 28519 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:45495
I20260925 02:30:03.472986 28653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:45495 every 8 connection(s)
I20260925 02:30:03.473272 28519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:30:03 dist-test-slave-dn4n krb5kdc[27741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:03.479367 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 28519
I20260925 02:30:03.479671 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:40099
--local_ip_for_outbound_sockets=127.11.220.195
--tserver_master_addrs=127.11.220.254:43577
--webserver_port=37563
--webserver_interface=127.11.220.195
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:30:03.481526 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.473868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:49241 (local address 127.11.220.254:43577)
0925 02:30:03.473937 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.473940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.474086 (+ 146us) server_negotiation.cc:407] Connection header received
0925 02:30:03.474565 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.474567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.474577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.474620 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:03.475416 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.475865 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.476741 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.476911 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.476953 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.478758 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:03.478760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:03.478760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:03.478767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:03.480219 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:03.480603 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:03.480604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:03.480605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:03.480661 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:03.480858 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:03.480859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:03.480859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:03.480990 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:03.481059 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.481404 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.481464 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:30:03.481777 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.473842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:49241)
0925 02:30:03.473990 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:03.474021 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:03.474463 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:03.474674 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:03.474689 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:03.475184 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:03.475350 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.475354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.475936 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.475937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.476675 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.476678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.476967 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.476967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.477055 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.477448 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:03.477455 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:03.478680 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:03.480288 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:03.480295 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:03.480296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:03.480543 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:03.480721 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:03.480721 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:03.480721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:03.480807 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:03.481090 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:03.481099 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:03.481263 (+ 164us) client_negotiation.cc:769] Sending connection context
0925 02:30:03.481375 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 02:30:03.481532 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":91,"threads_started":1}
I20260925 02:30:03.482298 28654 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43577
I20260925 02:30:03.482383 28654 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:03.482534 28654 heartbeater.cc:511] Master 127.11.220.254:43577 requested a full tablet report, sending...
I20260925 02:30:03.482923 28325 ts_manager.cc:194] Registered new tserver with Master: 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495)
I20260925 02:30:03.483299 28325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:49241
W20260925 02:30:03.562496 28657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:03.562697 28657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:03.562732 28657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:03.563840 28657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:03.563895 28657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:03.563938 28657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:30:03.565316 28657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:40099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=37563
--tserver_master_addrs=127.11.220.254:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:03.565551 28657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:03.565745 28657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:03.570123 28663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:03.570150 28665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:03.570178 28657 server_base.cc:1034] running on GCE node
W20260925 02:30:03.570147 28662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:03.570484 28657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:03.570705 28657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:03.571849 28657 hybrid_clock.cc:648] HybridClock initialized: now 1790303403571840 us; error 35 us; skew 500 ppm
Sep 25 02:30:03 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:03.573959 28657 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:30:03.574283 28657 webserver.cc:460] Webserver started at http://127.11.220.195:37563/ using document root <none> and password file <none>
I20260925 02:30:03.574474 28657 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:03.574539 28657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:03.575623 28657 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:03.576090 28672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:03.576232 28657 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:03.576301 28657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "7cde30baee774406bfed9ec5cde16213"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "4b4f530182849b16f4ba90b5295f7dd0"
server_key_iv: "0b1cb24f385fdbf8249c2405a6cf74d2"
server_key_version: "encryptionkey@0"
I20260925 02:30:03.576596 28657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:03.587809 28657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:03.589901 28657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:03.589995 28657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:03.590152 28657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:03.590431 28657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:03.590461 28657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:03.590494 28657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:03.590518 28657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:03.595615 28657 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:40099
I20260925 02:30:03.595657 28788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:40099 every 8 connection(s)
I20260925 02:30:03.595947 28657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:30:03 dist-test-slave-dn4n krb5kdc[27741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:03.603323 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.596609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:41073 (local address 127.11.220.254:43577)
0925 02:30:03.596650 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.596652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.596837 (+ 185us) server_negotiation.cc:407] Connection header received
0925 02:30:03.597220 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.597221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.597232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.597277 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:03.598031 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.598493 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.599357 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.599530 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.599578 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.601377 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:03.601378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:03.601378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:03.601384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:03.602401 (+ 1017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:03.602714 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:03.602715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:03.602715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:03.602759 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:03.602905 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:03.602905 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:03.602905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:03.602988 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:03.603031 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.603210 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.603264 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:30:03.603474 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.596604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:41073)
0925 02:30:03.596729 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:03.596758 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:03.597137 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:03.597322 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:03.597325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:03.597791 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:03.597947 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.597951 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.598569 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.598570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.599289 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.599290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.599592 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.599593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.599684 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.600059 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:03.600067 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:03.601304 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:03.602461 (+ 1157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:03.602467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:03.602468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:03.602659 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:03.602795 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:03.602796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:03.602796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:03.602865 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:03.603031 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:03.603033 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:03.603121 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 02:30:03.603187 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 02:30:03.603280 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
I20260925 02:30:03.603886 28789 heartbeater.cc:347] Connected to a master server at 127.11.220.254:43577
I20260925 02:30:03.603960 28789 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:03.604096 28789 heartbeater.cc:511] Master 127.11.220.254:43577 requested a full tablet report, sending...
I20260925 02:30:03.604414 28325 ts_manager.cc:194] Registered new tserver with Master: 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099)
I20260925 02:30:03.604739 28325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:41073
I20260925 02:30:03.604982 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 28657
I20260925 02:30:03.610198 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.605341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.11.220.254:43577)
0925 02:30:03.605377 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.605378 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.605552 (+ 174us) server_negotiation.cc:407] Connection header received
0925 02:30:03.605600 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.605601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.605608 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.605647 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:03.606141 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.606515 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.607131 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.607278 (+ 147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.607319 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.608358 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:03.608359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:03.608359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:03.608365 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:03.609403 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:03.609670 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:03.609671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:03.609671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:03.609712 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:03.609814 (+ 102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:03.609814 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:03.609814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:03.609881 (+ 67us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:03.609996 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.610026 (+ 30us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.610083 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:30:03.610930 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:30:03.613117 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.611258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.11.220.254:43577)
0925 02:30:03.611290 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.611292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.611408 (+ 116us) server_negotiation.cc:407] Connection header received
0925 02:30:03.611452 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.611452 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.611480 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.611533 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:30:03.611595 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.611961 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.612568 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.612717 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.612762 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.612871 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:30:03.612943 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:30:03.612993 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.613019 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.613031 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:30:03.613402 28325 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:30:03.614981 28325 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:30:03.616948 28448 tablet_service.cc:1467] Processing CreateTablet for tablet eb540fa863764d1ca6b2c9e75f5f4b21 (DEFAULT_TABLE table=test-table [id=45657167cbf94843965aa4c738b727c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:03.617148 28448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb540fa863764d1ca6b2c9e75f5f4b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:03.619681 28405 tablet_bootstrap.cc:492] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Bootstrap starting.
I20260925 02:30:03.620029 28405 tablet_bootstrap.cc:654] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:03.620412 28794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.616743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40099 (local address 127.0.0.1:41074)
0925 02:30:03.617028 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:03.617039 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:03.617091 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:03.617297 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:03.617297 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:03.617299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:03.617340 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.617342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.618325 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.618326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.619350 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.619353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.620176 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.620177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.620239 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.620258 (+ 19us) client_negotiation.cc:769] Sending connection context
0925 02:30:03.620310 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 02:30:03.620347 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":56,"threads_started":1}
I20260925 02:30:03.620579 28405 tablet_bootstrap.cc:492] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: No bootstrap required, opened a new log
I20260925 02:30:03.620628 28405 ts_tablet_manager.cc:1397] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:03.621016 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.616741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52568 (local address 127.11.220.194:45495)
0925 02:30:03.616920 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.616925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.616935 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:30:03.616974 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.616976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.616991 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.617248 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:03.617339 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.618545 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.619651 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.620630 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.620719 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.620781 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.620925 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.620956 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260925 02:30:03.621024 28523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.616637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:45495 (local address 127.0.0.1:52568)
0925 02:30:03.616681 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:03.616692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:03.616727 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:03.617130 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:03.617131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:03.617132 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:03.617173 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.617175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.618631 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.618632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.619571 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.619573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.620747 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.620748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.620835 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.620839 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:30:03.620914 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:30:03.620957 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 02:30:03.620981 28405 raft_consensus.cc:348] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:03.621059 28405 raft_consensus.cc:374] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:03.621124 28405 raft_consensus.cc:729] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e43809508c455581624aa26756ff17, State: Initialized, Role: FOLLOWER
I20260925 02:30:03.621193 28405 consensus_queue.cc:260] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:03.621295 28405 ts_tablet_manager.cc:1428] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 02:30:03.621397 28796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.616876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.11.220.195:40099)
0925 02:30:03.617017 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.617021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.617143 (+ 122us) server_negotiation.cc:407] Connection header received
0925 02:30:03.617178 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.617180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.617193 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.617272 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:03.617389 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.618249 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.619430 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.620039 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.620100 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.620144 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.620329 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.620375 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260925 02:30:03.621788 28588 tablet_service.cc:1467] Processing CreateTablet for tablet eb540fa863764d1ca6b2c9e75f5f4b21 (DEFAULT_TABLE table=test-table [id=45657167cbf94843965aa4c738b727c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:03.621939 28721 tablet_service.cc:1467] Processing CreateTablet for tablet eb540fa863764d1ca6b2c9e75f5f4b21 (DEFAULT_TABLE table=test-table [id=45657167cbf94843965aa4c738b727c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:03.622125 28588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb540fa863764d1ca6b2c9e75f5f4b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:03.622171 28721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb540fa863764d1ca6b2c9e75f5f4b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:03.624439 28798 tablet_bootstrap.cc:492] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: Bootstrap starting.
I20260925 02:30:03.624588 28799 tablet_bootstrap.cc:492] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: Bootstrap starting.
I20260925 02:30:03.625202 28798 tablet_bootstrap.cc:654] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:03.625204 28799 tablet_bootstrap.cc:654] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:03.625607 28799 log.cc:824] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:03.625666 28798 log.cc:824] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:03.626274 28799 tablet_bootstrap.cc:492] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: No bootstrap required, opened a new log
I20260925 02:30:03.626323 28799 ts_tablet_manager.cc:1397] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:03.626335 28798 tablet_bootstrap.cc:492] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: No bootstrap required, opened a new log
I20260925 02:30:03.626399 28798 ts_tablet_manager.cc:1397] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 02:30:03.627810 28799 raft_consensus.cc:348] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:03.627924 28799 raft_consensus.cc:374] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:03.627961 28799 raft_consensus.cc:729] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4efb5d48d94633bac21729213578d7, State: Initialized, Role: FOLLOWER
I20260925 02:30:03.627825 28798 raft_consensus.cc:348] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:03.628001 28798 raft_consensus.cc:374] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:03.628078 28798 raft_consensus.cc:729] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cde30baee774406bfed9ec5cde16213, State: Initialized, Role: FOLLOWER
I20260925 02:30:03.628084 28799 consensus_queue.cc:260] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:03.628202 28798 consensus_queue.cc:260] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:03.628368 28799 ts_tablet_manager.cc:1428] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:30:03.628391 28654 heartbeater.cc:503] Master 127.11.220.254:43577 was elected leader, sending a full tablet report...
I20260925 02:30:03.628469 28798 ts_tablet_manager.cc:1428] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:30:03.628592 28789 heartbeater.cc:503] Master 127.11.220.254:43577 was elected leader, sending a full tablet report...
W20260925 02:30:03.723839 28655 tablet.cc:2369] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:30:03.846436 28790 tablet.cc:2369] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:30:03.854442 28516 tablet.cc:2369] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:30:03.904057 28521 raft_consensus.cc:482] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:03.904182 28521 raft_consensus.cc:504] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:03.904552 28521 leader_election.cc:288] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495), 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099)
I20260925 02:30:03.907367 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.904810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:52819 (local address 127.11.220.194:45495)
0925 02:30:03.904873 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.904875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.904919 (+ 44us) server_negotiation.cc:407] Connection header received
0925 02:30:03.905077 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.905078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.905086 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.905150 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:03.905293 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.905746 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.906643 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.907003 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.907094 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.907110 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.907258 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.907297 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 02:30:03.907382 28796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.904985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:58787 (local address 127.11.220.195:40099)
0925 02:30:03.905033 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.905036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.905089 (+ 53us) server_negotiation.cc:407] Connection header received
0925 02:30:03.905158 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.905159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.905173 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.905233 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:03.905374 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.905813 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.906658 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.907016 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.907116 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.907124 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.907277 (+ 153us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.907315 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 02:30:03.907403 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.904706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:45495 (local address 127.11.220.193:52819)
0925 02:30:03.904853 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:03.904867 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:03.904952 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:03.905168 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:03.905169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:03.905171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:03.905226 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.905227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.905824 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.905826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.906539 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.906540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.907093 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.907094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.907154 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.907184 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 02:30:03.907252 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 02:30:03.907293 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260925 02:30:03.907406 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.904908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40099 (local address 127.11.220.193:58787)
0925 02:30:03.905008 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:03.905017 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:03.905103 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:03.905237 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:03.905238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:03.905240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:03.905296 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.905298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.905887 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.905888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.906572 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:03.906573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.907115 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.907116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:03.907176 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.907210 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 02:30:03.907271 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 02:30:03.907314 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":37,"threads_started":1}
I20260925 02:30:03.907827 28741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb540fa863764d1ca6b2c9e75f5f4b21" candidate_uuid: "77e43809508c455581624aa26756ff17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cde30baee774406bfed9ec5cde16213" is_pre_election: true
I20260925 02:30:03.907832 28608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb540fa863764d1ca6b2c9e75f5f4b21" candidate_uuid: "77e43809508c455581624aa26756ff17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4efb5d48d94633bac21729213578d7" is_pre_election: true
I20260925 02:30:03.907975 28608 raft_consensus.cc:2454] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77e43809508c455581624aa26756ff17 in term 0.
I20260925 02:30:03.907975 28741 raft_consensus.cc:2454] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77e43809508c455581624aa26756ff17 in term 0.
I20260925 02:30:03.908277 28400 leader_election.cc:302] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e43809508c455581624aa26756ff17, 7cde30baee774406bfed9ec5cde16213; no voters:
I20260925 02:30:03.908403 28521 raft_consensus.cc:2781] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:30:03.908500 28521 raft_consensus.cc:482] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:30:03.908535 28521 raft_consensus.cc:3037] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:03.909339 28521 raft_consensus.cc:504] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:03.909461 28521 leader_election.cc:288] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 1 election: Requested vote from peers 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495), 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099)
I20260925 02:30:03.909662 28608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb540fa863764d1ca6b2c9e75f5f4b21" candidate_uuid: "77e43809508c455581624aa26756ff17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4efb5d48d94633bac21729213578d7"
I20260925 02:30:03.909709 28741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb540fa863764d1ca6b2c9e75f5f4b21" candidate_uuid: "77e43809508c455581624aa26756ff17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cde30baee774406bfed9ec5cde16213"
I20260925 02:30:03.909739 28608 raft_consensus.cc:3037] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:03.909811 28741 raft_consensus.cc:3037] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:03.910456 28608 raft_consensus.cc:2454] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e43809508c455581624aa26756ff17 in term 1.
I20260925 02:30:03.910516 28741 raft_consensus.cc:2454] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e43809508c455581624aa26756ff17 in term 1.
I20260925 02:30:03.910640 28400 leader_election.cc:302] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b4efb5d48d94633bac21729213578d7, 77e43809508c455581624aa26756ff17; no voters:
I20260925 02:30:03.910733 28521 raft_consensus.cc:2781] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:03.910832 28521 raft_consensus.cc:686] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 LEADER]: Becoming Leader. State: Replica: 77e43809508c455581624aa26756ff17, State: Running, Role: LEADER
I20260925 02:30:03.910881 28521 consensus_queue.cc:237] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:03.911587 28324 catalog_manager.cc:5495] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77e43809508c455581624aa26756ff17 (127.11.220.193). New cstate: current_term: 1 leader_uuid: "77e43809508c455581624aa26756ff17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } health_report { overall_health: HEALTHY } } }
I20260925 02:30:03.974545 28809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:03.971684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38128 (local address 127.11.220.193:44041)
0925 02:30:03.971819 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:03.971821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:03.971828 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:30:03.971865 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:03.971865 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:03.971883 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:03.971948 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:30:03.972116 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.972620 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.973814 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:03.974015 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:03.974065 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:03.974258 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:30:03.974355 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:30:03.974405 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:03.974472 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 02:30:03.974488 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260925 02:30:03.976317 28608 raft_consensus.cc:1261] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7 [term 1 FOLLOWER]: Refusing update from remote peer 77e43809508c455581624aa26756ff17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:03.976317 28741 raft_consensus.cc:1261] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213 [term 1 FOLLOWER]: Refusing update from remote peer 77e43809508c455581624aa26756ff17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:03.976689 28806 consensus_queue.cc:1035] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:03.976851 28521 consensus_queue.cc:1035] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:03.979702 28810 mvcc.cc:205] Tried to move back new op lower bound from 7333082742684594176 to 7333082742419529728. Current Snapshot: MvccSnapshot[applied={T|T < 7333082742684594176}]
I20260925 02:30:03.979815 28811 mvcc.cc:205] Tried to move back new op lower bound from 7333082742684594176 to 7333082742419529728. Current Snapshot: MvccSnapshot[applied={T|T < 7333082742684594176}]
I20260925 02:30:03.980845 28812 mvcc.cc:205] Tried to move back new op lower bound from 7333082742684594176 to 7333082742419529728. Current Snapshot: MvccSnapshot[applied={T|T < 7333082742684594176}]
I20260925 02:30:03.986241 28325 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:03.986346 28325 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:03.986984 28325 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Sending DeleteTablet for 3 replicas of tablet eb540fa863764d1ca6b2c9e75f5f4b21
I20260925 02:30:03.987270 28588 tablet_service.cc:1514] Processing DeleteTablet for tablet eb540fa863764d1ca6b2c9e75f5f4b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:03 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:52568
I20260925 02:30:03.987334 28721 tablet_service.cc:1514] Processing DeleteTablet for tablet eb540fa863764d1ca6b2c9e75f5f4b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:03 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:41074
I20260925 02:30:03.987537 28448 tablet_service.cc:1514] Processing DeleteTablet for tablet eb540fa863764d1ca6b2c9e75f5f4b21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:03 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:38116
I20260925 02:30:03.987532 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 28363
I20260925 02:30:03.987603 28821 tablet_replica.cc:330] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: stopping tablet replica
I20260925 02:30:03.988654 28820 tablet_replica.cc:330] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: stopping tablet replica
I20260925 02:30:03.988744 28821 raft_consensus.cc:2229] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:03.988756 28820 raft_consensus.cc:2229] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:03.988853 28821 raft_consensus.cc:2258] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:03.988857 28820 raft_consensus.cc:2258] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:03.989527 28821 ts_tablet_manager.cc:1905] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:03.991386 28821 ts_tablet_manager.cc:1918] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:03.991446 28821 log.cc:1196] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/eb540fa863764d1ca6b2c9e75f5f4b21
I20260925 02:30:03.991686 28821 ts_tablet_manager.cc:1939] T eb540fa863764d1ca6b2c9e75f5f4b21 P 7cde30baee774406bfed9ec5cde16213: Deleting consensus metadata
I20260925 02:30:03.992230 28310 catalog_manager.cc:4841] TS 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099): tablet eb540fa863764d1ca6b2c9e75f5f4b21 (table test-table [id=45657167cbf94843965aa4c738b727c2]) successfully deleted
I20260925 02:30:03.996770 28820 ts_tablet_manager.cc:1905] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 02:30:03.997685 28311 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:44041 (error 108)
W20260925 02:30:03.998402 28523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:30:03.998038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:44041 (local address 127.0.0.1:38138)
0925 02:30:03.998120 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:03.998139 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:03.998186 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:03.998342 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:44041: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":61}
I20260925 02:30:03.998466 28820 ts_tablet_manager.cc:1918] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:03.998545 28820 log.cc:1196] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/eb540fa863764d1ca6b2c9e75f5f4b21
I20260925 02:30:03.998812 28820 ts_tablet_manager.cc:1939] T eb540fa863764d1ca6b2c9e75f5f4b21 P 6b4efb5d48d94633bac21729213578d7: Deleting consensus metadata
W20260925 02:30:03.998523 28311 catalog_manager.cc:4584] TS 77e43809508c455581624aa26756ff17 (127.11.220.193:44041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eb540fa863764d1ca6b2c9e75f5f4b21: Network error: Client connection negotiation failed: client connection to 127.11.220.193:44041: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 02:30:03.999184 28310 catalog_manager.cc:4841] TS 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495): tablet eb540fa863764d1ca6b2c9e75f5f4b21 (table test-table [id=45657167cbf94843965aa4c738b727c2]) successfully deleted
I20260925 02:30:03.999419 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 28519
I20260925 02:30:04.003801 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 28657
I20260925 02:30:04.009150 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 28292
I20260925 02:30:24.014599 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43577
--webserver_interface=127.11.220.254
--webserver_port=40245
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:43577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:24.090350 28823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:24.090523 28823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:24.090543 28823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:24.091527 28823 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 02:30:24.091554 28823 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 02:30:24.091565 28823 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 02:30:24.091576 28823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:24.091588 28823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:24.091598 28823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:24.091616 28823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:24.091627 28823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:24.093031 28823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:43577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:43577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=40245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:24.093216 28823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:24.093377 28823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:24.097739 28831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:24.097764 28828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:24.097740 28829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:24.097851 28823 server_base.cc:1034] running on GCE node
I20260925 02:30:24.098150 28823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:24.098455 28823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:24.099613 28823 hybrid_clock.cc:648] HybridClock initialized: now 1790303424099587 us; error 45 us; skew 500 ppm
Sep 25 02:30:24 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303424, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:24.101933 28823 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:30:24.102355 28823 webserver.cc:460] Webserver started at http://127.11.220.254:40245/ using document root <none> and password file <none>
I20260925 02:30:24.102545 28823 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:24.102602 28823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:24.103736 28823 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:24.104311 28838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:24.104547 28823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:24.104671 28823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "67b5ba0c9c644455895bf173fe297cb1"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "108cac9179fc7a61cc697e2750ffdb56"
server_key_iv: "376b58385d556302965fdd5d4e4de1ed"
server_key_version: "encryptionkey@0"
I20260925 02:30:24.104992 28823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:24.118342 28823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:24.120528 28823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:24.120671 28823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:24.124161 28823 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:43577
I20260925 02:30:24.124204 28891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:43577 every 8 connection(s)
I20260925 02:30:24.124611 28823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:24.125761 28893 sys_catalog.cc:263] Verifying existing consensus state
I20260925 02:30:24.126394 28893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Bootstrap starting.
I20260925 02:30:24.129189 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 28823
I20260925 02:30:24.129493 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:44041
--local_ip_for_outbound_sockets=127.11.220.193
--tserver_master_addrs=127.11.220.254:43577
--webserver_port=38785
--webserver_interface=127.11.220.193
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:30:24.135661 28893 log.cc:824] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:24.137622 28893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:30:24.137887 28893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: Bootstrap complete.
I20260925 02:30:24.139870 28893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:30:24.140108 28893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67b5ba0c9c644455895bf173fe297cb1, State: Initialized, Role: FOLLOWER
I20260925 02:30:24.140259 28893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:30:24.140313 28893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:24.140369 28893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:24.140411 28893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 2 FOLLOWER]: Advancing to term 3
I20260925 02:30:24.141330 28893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:30:24.141458 28893 leader_election.cc:302] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67b5ba0c9c644455895bf173fe297cb1; no voters:
I20260925 02:30:24.141637 28893 leader_election.cc:288] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 02:30:24.141667 28897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 02:30:24.141770 28897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [term 3 LEADER]: Becoming Leader. State: Replica: 67b5ba0c9c644455895bf173fe297cb1, State: Running, Role: LEADER
I20260925 02:30:24.141872 28893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:24.141899 28897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } }
I20260925 02:30:24.142361 28899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67b5ba0c9c644455895bf173fe297cb1. Latest consensus state: current_term: 3 leader_uuid: "67b5ba0c9c644455895bf173fe297cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } } }
I20260925 02:30:24.142452 28899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:24.142824 28898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "67b5ba0c9c644455895bf173fe297cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67b5ba0c9c644455895bf173fe297cb1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 43577 } } }
I20260925 02:30:24.142918 28898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:24.143741 28914 catalog_manager.cc:1259] Loaded cluster ID: 3e56a4535ae34a06a6b26b6cbf22e161
I20260925 02:30:24.143787 28914 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: loading cluster ID for follower catalog manager: success
I20260925 02:30:24.144829 28914 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 67b5ba0c9c644455895bf173fe297cb1: acquiring CA information for follower catalog manager: success
F20260925 02:30:24.145165 28914 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3711a629ad google::LogMessage::Fail() at ??:0
@ 0x7f3711a6588a google::LogMessage::SendToLog() at ??:0
@ 0x7f3711a624de google::LogMessage::Flush() at ??:0
@ 0x7f3711a63a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f37141cf104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f37199bb6b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f37199bb989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f37199bbe4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f37199be8c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f3713925954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f37147896db start_thread at ??:0
@ 0x7f371056061f clone at ??:0
W20260925 02:30:24.216661 28895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:24.216827 28895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:24.216849 28895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:24.217890 28895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:24.217923 28895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:24.217947 28895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:30:24.219198 28895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:44041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=38785
--tserver_master_addrs=127.11.220.254:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:24.219390 28895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:24.219604 28895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:24.223973 28932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:24.223991 28935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:24.224159 28933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:24.224050 28895 server_base.cc:1034] running on GCE node
I20260925 02:30:24.224359 28895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:24.224597 28895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:24.225751 28895 hybrid_clock.cc:648] HybridClock initialized: now 1790303424225739 us; error 28 us; skew 500 ppm
Sep 25 02:30:24 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303424, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:24.228166 28895 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:30:24.228682 28895 webserver.cc:460] Webserver started at http://127.11.220.193:38785/ using document root <none> and password file <none>
I20260925 02:30:24.228893 28895 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:24.228960 28895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:24.230170 28895 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:24.230832 28942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:24.231071 28895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:24.231170 28895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "77e43809508c455581624aa26756ff17"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "01a157f7725873bc2538bb0461579795"
server_key_iv: "11c4bad8854053304fae7767846570f7"
server_key_version: "encryptionkey@0"
I20260925 02:30:24.231458 28895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:24.240506 28895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:24.242523 28895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:24.242669 28895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:24.242885 28895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:24.243374 28949 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 02:30:24.245105 28895 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 02:30:24.245157 28895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 02:30:24.245193 28895 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 02:30:24.245736 28895 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 02:30:24.245798 28895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 02:30:24.245858 28949 tablet_bootstrap.cc:492] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Bootstrap starting.
I20260925 02:30:24.252600 28895 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:44041
I20260925 02:30:24.252722 29058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:44041 every 8 connection(s)
I20260925 02:30:24.252965 28895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:30:24.254395 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 28895
I20260925 02:30:24.254735 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:45495
--local_ip_for_outbound_sockets=127.11.220.194
--tserver_master_addrs=127.11.220.254:43577
--webserver_port=37525
--webserver_interface=127.11.220.194
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:30:24.257114 28949 log.cc:824] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:24.259083 28949 tablet_bootstrap.cc:492] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 02:30:24.259331 28949 tablet_bootstrap.cc:492] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Bootstrap complete.
I20260925 02:30:24.260277 28949 ts_tablet_manager.cc:1397] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.010s
I20260925 02:30:24.261301 28949 raft_consensus.cc:348] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:24.261548 28949 raft_consensus.cc:729] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e43809508c455581624aa26756ff17, State: Initialized, Role: FOLLOWER
I20260925 02:30:24.261683 28949 consensus_queue.cc:260] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:24.261940 28949 ts_tablet_manager.cc:1428] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
W20260925 02:30:24.327008 29061 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:30:24.253705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:54781)
0925 02:30:24.253880 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:24.253913 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:24.254422 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:24.326782 (+ 72360us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
W20260925 02:30:24.327279 29059 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:43577 (0 consecutive failures): Network error: Failed to ping master at 127.11.220.254:43577: Client connection negotiation failed: client connection to 127.11.220.254:43577: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 02:30:24.327625 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:24.327502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:38649)
0925 02:30:24.327559 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:24.327599 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 02:30:24.327993 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:24.327915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:39389)
0925 02:30:24.327956 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:24.327978 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 02:30:24.328132 29059 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 02:30:24.342048 29062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:24.342199 29062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:24.342222 29062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:24.343392 29062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:24.343428 29062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:24.343477 29062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:30:24.344900 29062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:45495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=37525
--tserver_master_addrs=127.11.220.254:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:24.345119 29062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:24.345326 29062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:24.349841 29069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:24.349850 29072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:24.350015 29062 server_base.cc:1034] running on GCE node
W20260925 02:30:24.349851 29070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:24.350270 29062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:24.350517 29062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:24.351662 29062 hybrid_clock.cc:648] HybridClock initialized: now 1790303424351656 us; error 35 us; skew 500 ppm
Sep 25 02:30:24 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303424, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:24.354125 29062 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:30:24.354502 29062 webserver.cc:460] Webserver started at http://127.11.220.194:37525/ using document root <none> and password file <none>
I20260925 02:30:24.354708 29062 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:24.354751 29062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:24.355975 29062 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:30:24.356657 29080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:24.356805 29062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 02:30:24.356884 29062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "6b4efb5d48d94633bac21729213578d7"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "8fb07fa5019f8b605cccb0f1d1af9392"
server_key_iv: "7b133f70183cfbef06e5ef73f4353462"
server_key_version: "encryptionkey@0"
I20260925 02:30:24.357159 29062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:24.378582 29062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:24.380164 29062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:24.380262 29062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:24.380414 29062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:24.380704 29062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:24.380736 29062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:24.380755 29062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:24.380767 29062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:24.386327 29062 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:45495
I20260925 02:30:24.386394 29195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:45495 every 8 connection(s)
I20260925 02:30:24.386705 29062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:30:24.387666 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:24.387343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:35487)
0925 02:30:24.387481 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:24.387540 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
W20260925 02:30:24.388041 29196 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:43577 (0 consecutive failures): Network error: Failed to ping master at 127.11.220.254:43577: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
I20260925 02:30:24.388293 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:24.388213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:37051)
0925 02:30:24.388252 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:24.388275 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 02:30:24.388549 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:24.388485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:42869)
0925 02:30:24.388511 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:24.388536 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260925 02:30:24.388669 29196 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 02:30:24.390606 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 29062
I20260925 02:30:24.390872 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:40099
--local_ip_for_outbound_sockets=127.11.220.195
--tserver_master_addrs=127.11.220.254:43577
--webserver_port=37563
--webserver_interface=127.11.220.195
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:24.464273 29199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:24.464428 29199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:24.464463 29199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:24.465464 29199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:24.465497 29199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:24.465540 29199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:30:24.466796 29199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:40099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=37563
--tserver_master_addrs=127.11.220.254:43577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:24.466997 29199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:24.467192 29199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:24.471608 29205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:24.471645 29204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:24.471712 29199 server_base.cc:1034] running on GCE node
W20260925 02:30:24.471799 29207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:24.471974 29199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:24.472193 29199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:24.473335 29199 hybrid_clock.cc:648] HybridClock initialized: now 1790303424473320 us; error 31 us; skew 500 ppm
Sep 25 02:30:24 dist-test-slave-dn4n krb5kdc[27741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303424, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:24.475492 29199 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:30:24.475829 29199 webserver.cc:460] Webserver started at http://127.11.220.195:37563/ using document root <none> and password file <none>
I20260925 02:30:24.475982 29199 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:24.476016 29199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:24.477121 29199 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260925 02:30:24.477555 29214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:24.477701 29199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:24.477778 29199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "7cde30baee774406bfed9ec5cde16213"
format_stamp: "Formatted at 2026-09-25 02:29:42 on dist-test-slave-dn4n"
server_key: "4b4f530182849b16f4ba90b5295f7dd0"
server_key_iv: "0b1cb24f385fdbf8249c2405a6cf74d2"
server_key_version: "encryptionkey@0"
I20260925 02:30:24.478031 29199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:24.495808 29199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:24.497375 29199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:24.497509 29199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:24.497691 29199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:24.497931 29199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:24.497975 29199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:24.498009 29199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:24.498032 29199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:24.503111 29199 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:40099
I20260925 02:30:24.503159 29328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:40099 every 8 connection(s)
I20260925 02:30:24.503410 29199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:30:24.504208 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:24.503917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:53965)
0925 02:30:24.504046 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:24.504101 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
W20260925 02:30:24.504562 29329 heartbeater.cc:644] Failed to heartbeat to 127.11.220.254:43577 (0 consecutive failures): Network error: Failed to ping master at 127.11.220.254:43577: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
I20260925 02:30:24.504796 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:24.504716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:42585)
0925 02:30:24.504757 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:24.504777 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 02:30:24.505060 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:24.504977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:49285)
0925 02:30:24.505022 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:24.505043 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 02:30:24.505159 29329 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 02:30:24.505321 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 29199
I20260925 02:30:25.328637 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:25.328422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:36407)
0925 02:30:25.328544 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:25.328576 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260925 02:30:25.389199 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:25.389009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:49741)
0925 02:30:25.389123 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:25.389154 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
I20260925 02:30:25.505597 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:25.505420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:36185)
0925 02:30:25.505522 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:25.505552 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":47,"threads_started":1}
I20260925 02:30:25.546886 29336 raft_consensus.cc:482] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:25.547006 29336 raft_consensus.cc:504] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:25.547403 29336 leader_election.cc:288] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495), 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099)
Sep 25 02:30:25 dist-test-slave-dn4n krb5kdc[27741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303424, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.194@KRBTEST.COM
Sep 25 02:30:25 dist-test-slave-dn4n krb5kdc[27741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303424, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.195@KRBTEST.COM
I20260925 02:30:25.556838 29337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:25.547673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:58839 (local address 127.11.220.194:45495)
0925 02:30:25.547821 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:25.547834 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:25.547866 (+ 32us) server_negotiation.cc:407] Connection header received
0925 02:30:25.548285 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:25.548287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:25.548325 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:25.548612 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:25.549278 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:25.550025 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:25.550938 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:25.551299 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:25.551397 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:25.553570 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:25.553575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:25.553575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:25.553585 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:25.555211 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:25.555600 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:25.555604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:25.555604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:25.555687 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:25.555923 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:25.555924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:25.555924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:25.556224 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:25.556305 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:25.556601 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 02:30:25.556729 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
I20260925 02:30:25.556838 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:25.547563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:45495 (local address 127.11.220.193:58839)
0925 02:30:25.547619 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:25.547629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:25.547696 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:25.548543 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:25.548546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:25.548973 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:25.549159 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:25.549165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:25.550170 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:25.550175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:25.550856 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:25.550858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:25.551383 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:25.551384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:25.551489 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:25.552019 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:25.552026 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:25.553462 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:25.555306 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:25.555313 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:25.555314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:25.555511 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:25.555762 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:25.555763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:25.555763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:25.555861 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:25.556296 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:25.556302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:25.556424 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 02:30:25.556549 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 02:30:25.556713 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":45}
I20260925 02:30:25.556924 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:25.547687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40099 (local address 127.11.220.193:60461)
0925 02:30:25.547781 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:25.547789 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:25.547827 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:25.548778 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:25.548778 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:25.548982 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:25.549159 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:25.549165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:25.550155 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:25.550156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:25.550857 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:25.550859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:25.551398 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:25.551399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:25.551489 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:25.552085 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:25.552089 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:25.553792 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:25.555657 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:25.555658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:25.555658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:25.555816 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:25.556076 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:25.556077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:25.556077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:25.556171 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:25.556530 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:25.556531 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:25.556571 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 02:30:25.556662 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 02:30:25.556773 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":3,"thread_start_us":60,"threads_started":1}
I20260925 02:30:25.556965 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:25.547898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:60461 (local address 127.11.220.195:40099)
0925 02:30:25.548033 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:25.548043 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:25.548075 (+ 32us) server_negotiation.cc:407] Connection header received
0925 02:30:25.548614 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:25.548617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:25.548664 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:25.548790 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:25.549297 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:25.550023 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:25.550943 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:25.551305 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:25.551415 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:25.553913 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:25.553917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:25.553917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:25.553929 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:25.555558 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:25.555892 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:25.555895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:25.555895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:25.555996 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:25.556232 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:25.556232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:25.556233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:25.556434 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:25.556536 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:25.556738 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 02:30:25.556863 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
W20260925 02:30:25.557667 28944 leader_election.cc:341] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099): Not found: Tablet not found: eb540fa863764d1ca6b2c9e75f5f4b21
W20260925 02:30:25.558264 28944 leader_election.cc:341] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495): Not found: Tablet not found: eb540fa863764d1ca6b2c9e75f5f4b21
I20260925 02:30:25.558308 28944 leader_election.cc:302] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 77e43809508c455581624aa26756ff17; no voters: 6b4efb5d48d94633bac21729213578d7, 7cde30baee774406bfed9ec5cde16213
I20260925 02:30:25.558405 29336 raft_consensus.cc:2726] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:30:26.329152 29340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:26.328969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:40385)
0925 02:30:26.329081 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:26.329109 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260925 02:30:26.389778 29341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:26.389586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:47825)
0925 02:30:26.389711 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:26.389743 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":54,"threads_started":1}
I20260925 02:30:26.506181 29342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:26.505996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:39305)
0925 02:30:26.506104 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:26.506135 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":54,"threads_started":1}
I20260925 02:30:27.279492 29343 raft_consensus.cc:482] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:27.279574 29343 raft_consensus.cc:504] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:27.279695 29343 leader_election.cc:288] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495), 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099)
W20260925 02:30:27.280150 28944 leader_election.cc:341] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099): Not found: Tablet not found: eb540fa863764d1ca6b2c9e75f5f4b21
W20260925 02:30:27.280254 28944 leader_election.cc:341] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495): Not found: Tablet not found: eb540fa863764d1ca6b2c9e75f5f4b21
I20260925 02:30:27.280295 28944 leader_election.cc:302] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 77e43809508c455581624aa26756ff17; no voters: 6b4efb5d48d94633bac21729213578d7, 7cde30baee774406bfed9ec5cde16213
I20260925 02:30:27.280359 29343 raft_consensus.cc:2726] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:30:27.329713 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:27.329511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:33835)
0925 02:30:27.329628 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:27.329656 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260925 02:30:27.390558 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:27.390380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:39709)
0925 02:30:27.390500 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:27.390526 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260925 02:30:27.506698 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:27.506534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:54199)
0925 02:30:27.506633 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:27.506663 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":51,"threads_started":1}
I20260925 02:30:28.330295 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:28.330101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:50597)
0925 02:30:28.330220 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:28.330251 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260925 02:30:28.391145 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:28.390945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:34907)
0925 02:30:28.391077 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:28.391109 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260925 02:30:28.507283 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:28.507083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:43189)
0925 02:30:28.507205 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:28.507234 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260925 02:30:29.330982 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:29.330775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:43663)
0925 02:30:29.330905 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:29.330937 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260925 02:30:29.391685 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:29.391519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:54541)
0925 02:30:29.391623 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:29.391651 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":51,"threads_started":1}
I20260925 02:30:29.507844 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:29.507652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:53113)
0925 02:30:29.507784 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:29.507813 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260925 02:30:30.331557 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:30.331364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:48541)
0925 02:30:30.331484 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:30.331513 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260925 02:30:30.392328 29354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:30.392145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:42339)
0925 02:30:30.392250 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:30.392295 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260925 02:30:30.456113 29355 raft_consensus.cc:482] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:30.456210 29355 raft_consensus.cc:504] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:30.456373 29355 leader_election.cc:288] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495), 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099)
W20260925 02:30:30.456746 28944 leader_election.cc:341] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495): Not found: Tablet not found: eb540fa863764d1ca6b2c9e75f5f4b21
W20260925 02:30:30.456859 28944 leader_election.cc:341] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099): Not found: Tablet not found: eb540fa863764d1ca6b2c9e75f5f4b21
I20260925 02:30:30.456874 28944 leader_election.cc:302] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 77e43809508c455581624aa26756ff17; no voters: 6b4efb5d48d94633bac21729213578d7, 7cde30baee774406bfed9ec5cde16213
I20260925 02:30:30.456939 29355 raft_consensus.cc:2726] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:30:30.508490 29356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:30.508210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:34493)
0925 02:30:30.508406 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:30.508438 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":60,"threads_started":1}
I20260925 02:30:31.332132 29357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:31.331934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:45507)
0925 02:30:31.332057 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:31.332088 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260925 02:30:31.392889 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:31.392712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:35605)
0925 02:30:31.392810 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:31.392838 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":51,"threads_started":1}
I20260925 02:30:31.509099 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:31.508912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:43837)
0925 02:30:31.509034 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:31.509067 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260925 02:30:32.332757 29360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:32.332553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:54267)
0925 02:30:32.332677 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:32.332712 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260925 02:30:32.393385 29361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:32.393220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:53091)
0925 02:30:32.393325 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:32.393352 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
I20260925 02:30:32.469614 29362 raft_consensus.cc:482] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:32.469694 29362 raft_consensus.cc:504] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:32.469899 29362 leader_election.cc:288] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495), 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099)
W20260925 02:30:32.470252 28944 leader_election.cc:341] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495): Not found: Tablet not found: eb540fa863764d1ca6b2c9e75f5f4b21
W20260925 02:30:32.470321 28944 leader_election.cc:341] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099): Not found: Tablet not found: eb540fa863764d1ca6b2c9e75f5f4b21
I20260925 02:30:32.470355 28944 leader_election.cc:302] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 77e43809508c455581624aa26756ff17; no voters: 6b4efb5d48d94633bac21729213578d7, 7cde30baee774406bfed9ec5cde16213
I20260925 02:30:32.470418 29362 raft_consensus.cc:2726] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:30:32.509687 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:32.509504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:42425)
0925 02:30:32.509613 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:32.509643 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260925 02:30:33.333357 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:33.333171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:57997)
0925 02:30:33.333282 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:33.333313 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260925 02:30:33.394117 29365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:33.393948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:38009)
0925 02:30:33.394054 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:33.394084 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260925 02:30:33.510248 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:33.510073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:45255)
0925 02:30:33.510187 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:33.510217 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260925 02:30:34.334051 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:34.333838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:50255)
0925 02:30:34.333956 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:34.333990 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260925 02:30:34.394781 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:34.394581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:37333)
0925 02:30:34.394692 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:34.394747 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260925 02:30:34.510824 29369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:34.510644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:37589)
0925 02:30:34.510757 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:34.510788 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":49,"threads_started":1}
I20260925 02:30:35.334658 29370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:35.334462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:36725)
0925 02:30:35.334579 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:35.334612 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260925 02:30:35.395288 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:35.395126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:32953)
0925 02:30:35.395225 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:35.395254 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":51,"threads_started":1}
I20260925 02:30:35.511461 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:35.511263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:50391)
0925 02:30:35.511395 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:35.511427 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260925 02:30:36.335290 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:36.335058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:54797)
0925 02:30:36.335211 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:36.335244 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260925 02:30:36.395933 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:36.395718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:33369)
0925 02:30:36.395859 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:36.395894 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":56,"threads_started":1}
I20260925 02:30:36.512039 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:36.511856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:45361)
0925 02:30:36.511972 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:36.512003 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260925 02:30:36.956274 29376 raft_consensus.cc:482] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:36.956363 29376 raft_consensus.cc:504] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b4efb5d48d94633bac21729213578d7" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 45495 } } peers { permanent_uuid: "7cde30baee774406bfed9ec5cde16213" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40099 } } peers { permanent_uuid: "77e43809508c455581624aa26756ff17" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 44041 } }
I20260925 02:30:36.956533 29376 leader_election.cc:288] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495), 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099)
W20260925 02:30:36.956856 28944 leader_election.cc:341] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6b4efb5d48d94633bac21729213578d7 (127.11.220.194:45495): Not found: Tablet not found: eb540fa863764d1ca6b2c9e75f5f4b21
W20260925 02:30:36.956938 28944 leader_election.cc:341] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7cde30baee774406bfed9ec5cde16213 (127.11.220.195:40099): Not found: Tablet not found: eb540fa863764d1ca6b2c9e75f5f4b21
I20260925 02:30:36.956967 28944 leader_election.cc:302] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 77e43809508c455581624aa26756ff17; no voters: 6b4efb5d48d94633bac21729213578d7, 7cde30baee774406bfed9ec5cde16213
I20260925 02:30:36.957067 29376 raft_consensus.cc:2726] T eb540fa863764d1ca6b2c9e75f5f4b21 P 77e43809508c455581624aa26756ff17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 02:30:37.335935 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:37.335734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:44881)
0925 02:30:37.335859 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:37.335890 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260925 02:30:37.396564 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:37.396357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:44765)
0925 02:30:37.396494 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:37.396520 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260925 02:30:37.512639 29379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:37.512466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:58441)
0925 02:30:37.512575 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:37.512606 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":47,"threads_started":1}
I20260925 02:30:38.336689 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:38.336511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:42559)
0925 02:30:38.336614 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:38.336644 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":50,"threads_started":1}
I20260925 02:30:38.397101 29381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:38.396926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:57089)
0925 02:30:38.397037 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:38.397068 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":53,"threads_started":1}
I20260925 02:30:38.513177 29382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:38.513009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:32983)
0925 02:30:38.513112 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:38.513143 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":52,"threads_started":1}
I20260925 02:30:39.337323 29383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:39.337144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.193:52915)
0925 02:30:39.337252 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:39.337281 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
I20260925 02:30:39.397665 29384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:39.397496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.194:54209)
0925 02:30:39.397600 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:39.397626 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":52,"threads_started":1}
I20260925 02:30:39.507045 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 28895
I20260925 02:30:39.511835 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 29062
I20260925 02:30:39.513739 29385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:39.513567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:43577 (local address 127.11.220.195:39421)
0925 02:30:39.513664 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:39.513697 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.254:43577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":44,"threads_started":1}
I20260925 02:30:39.515887 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 29199
2026-09-25T02:30:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58101 ms)
[----------] 30 tests from SecurityITest (217280 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:39 dist-test-slave-dn4n krb5kdc[29388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:39 dist-test-slave-dn4n krb5kdc[29388](info): set up 2 sockets
Sep 25 02:30:39 dist-test-slave-dn4n krb5kdc[29388](info): commencing operation
krb5kdc: starting...
W20260925 02:30:40.500972 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:40 dist-test-slave-dn4n krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:30:40.604213 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:35113
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:35113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:40.679287 29401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:40.679448 29401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:40.679499 29401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:40.680636 29401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:40.680672 29401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:40.680686 29401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:40.680712 29401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:40.680724 29401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:40.682054 29401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:35113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:35113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:40.682219 29401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:40.682368 29401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:40.686700 29407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:40.686699 29406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:40.686821 29401 server_base.cc:1034] running on GCE node
W20260925 02:30:40.686713 29409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:40.687151 29401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:40.687431 29401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:40.688587 29401 hybrid_clock.cc:648] HybridClock initialized: now 1790303440688573 us; error 26 us; skew 500 ppm
Sep 25 02:30:40 dist-test-slave-dn4n krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:40.690896 29401 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:30:40.691265 29401 webserver.cc:460] Webserver started at http://127.11.220.254:35263/ using document root <none> and password file <none>
I20260925 02:30:40.691496 29401 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:40.691540 29401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:40.691666 29401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:40.692646 29401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "98e8c7b15e6d40fb96c48ce8e5045656"
format_stamp: "Formatted at 2026-09-25 02:30:40 on dist-test-slave-dn4n"
server_key: "bd480816d93f41fdbeadcc580b4406d1"
server_key_iv: "651b889b5566272f0bc0ea446573f0c9"
server_key_version: "encryptionkey@0"
I20260925 02:30:40.692960 29401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "98e8c7b15e6d40fb96c48ce8e5045656"
format_stamp: "Formatted at 2026-09-25 02:30:40 on dist-test-slave-dn4n"
server_key: "bd480816d93f41fdbeadcc580b4406d1"
server_key_iv: "651b889b5566272f0bc0ea446573f0c9"
server_key_version: "encryptionkey@0"
I20260925 02:30:40.694131 29401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:40.694751 29416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:40.694952 29401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:40.695050 29401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "98e8c7b15e6d40fb96c48ce8e5045656"
format_stamp: "Formatted at 2026-09-25 02:30:40 on dist-test-slave-dn4n"
server_key: "bd480816d93f41fdbeadcc580b4406d1"
server_key_iv: "651b889b5566272f0bc0ea446573f0c9"
server_key_version: "encryptionkey@0"
I20260925 02:30:40.695113 29401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:40.712783 29401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:40.714828 29401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:40.714926 29401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:40.718636 29401 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:35113
I20260925 02:30:40.718683 29469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:35113 every 8 connection(s)
I20260925 02:30:40.718974 29401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:40.719774 29470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:40.721906 29470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656: Bootstrap starting.
I20260925 02:30:40.722468 29470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:40.722759 29470 log.cc:824] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:40.723388 29470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656: No bootstrap required, opened a new log
I20260925 02:30:40.725055 29470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e8c7b15e6d40fb96c48ce8e5045656" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35113 } }
I20260925 02:30:40.725193 29470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:40.725214 29470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e8c7b15e6d40fb96c48ce8e5045656, State: Initialized, Role: FOLLOWER
I20260925 02:30:40.725387 29470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e8c7b15e6d40fb96c48ce8e5045656" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35113 } }
I20260925 02:30:40.725445 29470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:40.725485 29470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:40.725531 29470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:40.726014 29470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e8c7b15e6d40fb96c48ce8e5045656" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35113 } }
I20260925 02:30:40.726143 29470 leader_election.cc:302] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98e8c7b15e6d40fb96c48ce8e5045656; no voters:
I20260925 02:30:40.726310 29470 leader_election.cc:288] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:40.726357 29473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:40.726481 29473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [term 1 LEADER]: Becoming Leader. State: Replica: 98e8c7b15e6d40fb96c48ce8e5045656, State: Running, Role: LEADER
I20260925 02:30:40.726550 29470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:40.726605 29473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e8c7b15e6d40fb96c48ce8e5045656" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35113 } }
I20260925 02:30:40.726949 29476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98e8c7b15e6d40fb96c48ce8e5045656. Latest consensus state: current_term: 1 leader_uuid: "98e8c7b15e6d40fb96c48ce8e5045656" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e8c7b15e6d40fb96c48ce8e5045656" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35113 } } }
I20260925 02:30:40.727083 29475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98e8c7b15e6d40fb96c48ce8e5045656" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e8c7b15e6d40fb96c48ce8e5045656" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35113 } } }
I20260925 02:30:40.727120 29476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:40.727195 29475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:40.727488 29483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:40.727958 29483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:40.728423 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 29401
I20260925 02:30:40.728550 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:40.728744 12147 external_mini_cluster.cc:1428] Setting key 9762223cf3156bd79487e672216e2cfb
I20260925 02:30:40.730394 29483 catalog_manager.cc:1347] Generated new cluster ID: 0a31b545ff4c4094be2651a89238ddd0
I20260925 02:30:40.730443 29483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:30:40 dist-test-slave-dn4n krb5kdc[29388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:40.738101 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:40.729140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58798 (local address 127.11.220.254:35113)
0925 02:30:40.729709 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:40.729717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:40.729746 (+ 29us) server_negotiation.cc:407] Connection header received
0925 02:30:40.729846 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:40.729851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:40.729897 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:40.730017 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:40.730588 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:40.731462 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:40.732251 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:40.732744 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:40.732867 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:40.735064 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:40.735068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:40.735068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:40.735102 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:40.736683 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:40.737014 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:40.737017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:40.737017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:40.737106 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:40.737297 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:40.737298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:40.737298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:40.737500 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:40.737658 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:40.737744 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 02:30:40.737950 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":63,"threads_started":1}
I20260925 02:30:40.739477 29483 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:40.740111 29483 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:40.749035 29483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656: Generated new TSK 0
I20260925 02:30:40.749176 29483 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:30:40.779088 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35113
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:40.851275 29497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:40.851441 29497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:40.851480 29497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:40.852667 29497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:40.852701 29497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:40.852725 29497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:30:40.854022 29497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:40.854194 29497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:40.854403 29497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:40.858688 29505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:40.858739 29503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:40.858700 29502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:40.858973 29497 server_base.cc:1034] running on GCE node
I20260925 02:30:40.859127 29497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:40.859300 29497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:40.860496 29497 hybrid_clock.cc:648] HybridClock initialized: now 1790303440860479 us; error 41 us; skew 500 ppm
Sep 25 02:30:40 dist-test-slave-dn4n krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:40.862684 29497 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:30:40.863026 29497 webserver.cc:460] Webserver started at http://127.11.220.193:39361/ using document root <none> and password file <none>
I20260925 02:30:40.863232 29497 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:40.863292 29497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:40.863428 29497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:40.864319 29497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "1133a310c2a542a1a8013198c3f94b79"
format_stamp: "Formatted at 2026-09-25 02:30:40 on dist-test-slave-dn4n"
server_key: "fa6b304c58bd7805ad5c95a92846806c"
server_key_iv: "7bc63b1f83a23d543f183979433c6521"
server_key_version: "encryptionkey@0"
I20260925 02:30:40.864656 29497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "1133a310c2a542a1a8013198c3f94b79"
format_stamp: "Formatted at 2026-09-25 02:30:40 on dist-test-slave-dn4n"
server_key: "fa6b304c58bd7805ad5c95a92846806c"
server_key_iv: "7bc63b1f83a23d543f183979433c6521"
server_key_version: "encryptionkey@0"
I20260925 02:30:40.865869 29497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:30:40.866530 29512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:40.866688 29497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:40.866772 29497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "1133a310c2a542a1a8013198c3f94b79"
format_stamp: "Formatted at 2026-09-25 02:30:40 on dist-test-slave-dn4n"
server_key: "fa6b304c58bd7805ad5c95a92846806c"
server_key_iv: "7bc63b1f83a23d543f183979433c6521"
server_key_version: "encryptionkey@0"
I20260925 02:30:40.866843 29497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:40.891969 29497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:40.894464 29497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:40.894598 29497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:40.894865 29497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:40.895193 29497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:40.895267 29497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:40.895313 29497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:40.895331 29497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:40.902161 29497 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:38323
I20260925 02:30:40.902236 29629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:38323 every 8 connection(s)
I20260925 02:30:40.902516 29497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:30:40.902968 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 29497
I20260925 02:30:40.903065 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:30:40.903314 12147 external_mini_cluster.cc:1428] Setting key d0411a667297522f8776bf83026caa46
Sep 25 02:30:40 dist-test-slave-dn4n krb5kdc[29388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303440, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
I20260925 02:30:40.913512 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:40.903185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:57995 (local address 127.11.220.254:35113)
0925 02:30:40.903277 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:40.903280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:40.903466 (+ 186us) server_negotiation.cc:407] Connection header received
0925 02:30:40.904140 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:40.904141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:40.904152 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:40.904190 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:40.905347 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:40.906097 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:40.907342 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:40.907519 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:40.907608 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:40.910290 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:40.910292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:40.910292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:40.910300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:40.911879 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:40.912366 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:40.912367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:40.912367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:40.912416 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:40.912775 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:40.912776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:40.912776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:40.912915 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:40.913010 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:40.913346 (+ 336us) server_negotiation.cc:299] Negotiation successful
0925 02:30:40.913424 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:30:40.913674 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:40.903163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35113 (local address 127.11.220.193:57995)
0925 02:30:40.903347 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:40.903394 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:40.904042 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:40.904252 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:40.904256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:40.905002 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:40.905252 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:40.905258 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:40.906222 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:40.906224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:40.907263 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:40.907266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:40.907619 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:40.907621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:40.907737 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:40.908250 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:40.908261 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:40.910194 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:40.911965 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:40.911974 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:40.911976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:40.912283 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:40.912532 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:40.912533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:40.912533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:40.912706 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:40.913041 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:40.913048 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:40.913215 (+ 167us) client_negotiation.cc:769] Sending connection context
0925 02:30:40.913322 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 02:30:40.913461 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
I20260925 02:30:40.914075 29630 heartbeater.cc:347] Connected to a master server at 127.11.220.254:35113
I20260925 02:30:40.914179 29630 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:40.914369 29630 heartbeater.cc:511] Master 127.11.220.254:35113 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
I20260925 02:30:40.914956 29433 ts_manager.cc:194] Registered new tserver with Master: 1133a310c2a542a1a8013198c3f94b79 (127.11.220.193:38323)
I20260925 02:30:40.915894 29433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:57995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:30:40.929083 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35113
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:41.001081 29635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:41.001271 29635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:41.001291 29635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:41.002332 29635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:41.002363 29635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:41.002388 29635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:30:41.003690 29635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:41.003866 29635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:41.004015 29635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:41.008297 29641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:41.008340 29635 server_base.cc:1034] running on GCE node
W20260925 02:30:41.008307 29643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:41.008358 29640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:41.008620 29635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:41.008785 29635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:41.009924 29635 hybrid_clock.cc:648] HybridClock initialized: now 1790303441009902 us; error 37 us; skew 500 ppm
Sep 25 02:30:41 dist-test-slave-dn4n krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:41.012151 29635 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:30:41.012595 29635 webserver.cc:460] Webserver started at http://127.11.220.194:42459/ using document root <none> and password file <none>
I20260925 02:30:41.012822 29635 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:41.012862 29635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:41.012998 29635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:41.013883 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "edcaf0393ee7488bb00348496e711500"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "c821e65c3ad82f9a490f648436ee6dff"
server_key_iv: "b763accf66ab1d7238941d08b51da3b6"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.014189 29635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "edcaf0393ee7488bb00348496e711500"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "c821e65c3ad82f9a490f648436ee6dff"
server_key_iv: "b763accf66ab1d7238941d08b51da3b6"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.015255 29635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:30:41.015884 29650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.016083 29635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:41.016168 29635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "edcaf0393ee7488bb00348496e711500"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "c821e65c3ad82f9a490f648436ee6dff"
server_key_iv: "b763accf66ab1d7238941d08b51da3b6"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.016237 29635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:41.032291 29635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:41.034416 29635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:41.034559 29635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:41.034742 29635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:41.034992 29635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:41.035022 29635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.035070 29635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:41.035092 29635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.040160 29635 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:42691
I20260925 02:30:41.040223 29766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:42691 every 8 connection(s)
I20260925 02:30:41.040542 29635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:30:41.042379 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 29635
I20260925 02:30:41.042492 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:30:41.042716 12147 external_mini_cluster.cc:1428] Setting key e20bcc7610f205b063254eae1cc447d5
Sep 25 02:30:41 dist-test-slave-dn4n krb5kdc[29388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
I20260925 02:30:41.050238 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.041166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:45753 (local address 127.11.220.254:35113)
0925 02:30:41.041210 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.041213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.041411 (+ 198us) server_negotiation.cc:407] Connection header received
0925 02:30:41.041788 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.041789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.041799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.041845 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:41.042657 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.043211 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.045402 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.045583 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.045636 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.047651 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:41.047652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:41.047653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:41.047660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:41.048940 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:41.049355 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:41.049357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:41.049357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:41.049418 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:41.049647 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:41.049648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:41.049649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:41.049753 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:41.049822 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.050073 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.050158 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:30:41.050371 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.041157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35113 (local address 127.11.220.194:45753)
0925 02:30:41.041308 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.041337 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.041715 (+ 378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.041886 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.041888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.042356 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:41.042561 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.042566 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.044571 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.044572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.045316 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.045319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.045662 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.045663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.045802 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.046199 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:41.046208 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:41.047561 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:41.049050 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:41.049057 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:41.049059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:41.049271 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:41.049481 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:41.049482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:41.049482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:41.049585 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:41.049814 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:41.049820 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:41.049945 (+ 125us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.050041 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.050177 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260925 02:30:41.050767 29767 heartbeater.cc:347] Connected to a master server at 127.11.220.254:35113
I20260925 02:30:41.050834 29767 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:41.051010 29767 heartbeater.cc:511] Master 127.11.220.254:35113 requested a full tablet report, sending...
I20260925 02:30:41.051360 29433 ts_manager.cc:194] Registered new tserver with Master: edcaf0393ee7488bb00348496e711500 (127.11.220.194:42691)
I20260925 02:30:41.051733 29433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:45753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:30:41.064942 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35113
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:41.137456 29772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:41.137616 29772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:41.137636 29772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:41.138660 29772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:41.138696 29772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:41.138720 29772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:30:41.139998 29772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:41.140224 29772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:41.140408 29772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:41.144912 29777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:41.144920 29780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:41.144920 29778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:41.145133 29772 server_base.cc:1034] running on GCE node
I20260925 02:30:41.145306 29772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:41.145519 29772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:41.146667 29772 hybrid_clock.cc:648] HybridClock initialized: now 1790303441146662 us; error 31 us; skew 500 ppm
Sep 25 02:30:41 dist-test-slave-dn4n krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:41.148693 29772 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:30:41.149022 29772 webserver.cc:460] Webserver started at http://127.11.220.195:42683/ using document root <none> and password file <none>
I20260925 02:30:41.149183 29772 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:41.149215 29772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:41.149286 29772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:41.150048 29772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "5cdc9455d2be4e099c518867f638a978"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "8951bbf284c9f9728ff4a800268501d5"
server_key_iv: "6d185d8848d3bc949ad777906ced78b1"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.150293 29772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "5cdc9455d2be4e099c518867f638a978"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "8951bbf284c9f9728ff4a800268501d5"
server_key_iv: "6d185d8848d3bc949ad777906ced78b1"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.151304 29772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:30:41.151902 29787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.152055 29772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:41.152125 29772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "5cdc9455d2be4e099c518867f638a978"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "8951bbf284c9f9728ff4a800268501d5"
server_key_iv: "6d185d8848d3bc949ad777906ced78b1"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.152172 29772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:41.163620 29772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:41.165196 29772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:41.165284 29772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:41.165436 29772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:41.165623 29772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:41.165649 29772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.165666 29772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:41.165678 29772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.170962 29772 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:40451
I20260925 02:30:41.171022 29902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:40451 every 8 connection(s)
I20260925 02:30:41.171288 29772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:30:41 dist-test-slave-dn4n krb5kdc[29388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:41.178966 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 29772
I20260925 02:30:41.179098 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:30:41.179211 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.171940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:48033 (local address 127.11.220.254:35113)
0925 02:30:41.171980 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.171983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.172143 (+ 160us) server_negotiation.cc:407] Connection header received
0925 02:30:41.172596 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.172598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.172611 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.172669 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:41.173464 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.173919 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.174805 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.175025 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.175093 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.176996 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:41.176998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:41.176998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:41.177005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:41.178093 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:41.178442 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:41.178442 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:41.178443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:41.178489 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:41.178680 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:41.178681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:41.178681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:41.178781 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:41.178825 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.179041 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.179119 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:30:41.179306 12147 external_mini_cluster.cc:1428] Setting key a37b91d8aee3d358a5de822a0caf2bff
I20260925 02:30:41.179349 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.171895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35113 (local address 127.11.220.195:48033)
0925 02:30:41.172023 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.172068 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.172490 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.172729 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.172733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.173216 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:41.173370 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.173374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.174010 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.174011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.174712 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.174714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.175101 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.175102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.175217 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.175568 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:41.175575 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:41.176900 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:41.178158 (+ 1258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:41.178164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:41.178165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:41.178364 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:41.178545 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:41.178545 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:41.178546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:41.178630 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:41.178830 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:41.178833 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:41.178921 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.178991 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.179137 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260925 02:30:41.179833 29903 heartbeater.cc:347] Connected to a master server at 127.11.220.254:35113
I20260925 02:30:41.179945 29903 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:41.180096 29903 heartbeater.cc:511] Master 127.11.220.254:35113 requested a full tablet report, sending...
I20260925 02:30:41.180444 29433 ts_manager.cc:194] Registered new tserver with Master: 5cdc9455d2be4e099c518867f638a978 (127.11.220.195:40451)
I20260925 02:30:41.180790 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:30:41.180832 29433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:48033
I20260925 02:30:41.187170 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.182090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58808 (local address 127.11.220.254:35113)
0925 02:30:41.182141 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.182142 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.182212 (+ 70us) server_negotiation.cc:407] Connection header received
0925 02:30:41.182251 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.182252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.182258 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.182304 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:41.182795 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.183210 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.183863 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.184015 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.184058 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.185108 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:41.185109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:41.185109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:41.185115 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:41.186380 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:41.186640 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:41.186640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:41.186641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:41.186697 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:41.186830 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:41.186831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:41.186831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:41.186939 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:41.187001 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.187072 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.187121 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 02:30:41.188587 29433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:30:41.190150 29433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:30:41.196298 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.192394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:40451 (local address 127.0.0.1:35784)
0925 02:30:41.192939 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.192947 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.192996 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.193216 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.193217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.193219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:41.193256 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.193257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.194037 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.194038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.195218 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.195221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.196013 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.196014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.196084 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.196102 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.196179 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.196229 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":21,"thread_start_us":83,"threads_started":1}
I20260925 02:30:41.196313 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.192486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35784 (local address 127.11.220.195:40451)
0925 02:30:41.192627 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.192631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.193052 (+ 421us) server_negotiation.cc:407] Connection header received
0925 02:30:41.193087 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.193089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.193103 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.193164 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:41.193323 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.193928 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.195285 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.195928 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.195999 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.196043 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.196208 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.196248 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":94,"threads_started":1}
I20260925 02:30:41.196381 29911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.192203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:42691 (local address 127.0.0.1:36698)
0925 02:30:41.192304 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.192335 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.192389 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.192897 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.192899 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.192901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:41.192970 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.192971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.194245 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.194246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.195357 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.195359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.196154 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.196155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.196205 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.196208 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.196257 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.196285 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":23,"thread_start_us":61,"threads_started":1}
I20260925 02:30:41.196388 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.192559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.11.220.194:42691)
0925 02:30:41.192720 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.192724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.192734 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:30:41.192770 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.192772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.192787 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.192865 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:41.193067 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.194122 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.195436 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.196082 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.196140 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.196197 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.196269 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.196304 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260925 02:30:41.197046 29700 tablet_service.cc:1467] Processing CreateTablet for tablet 809b0ade4abf4dc89db255dc3f5fd9de (DEFAULT_TABLE table=test-table [id=99f3a4913be54af08ba9edf3a7f98bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:41.197041 29836 tablet_service.cc:1467] Processing CreateTablet for tablet 809b0ade4abf4dc89db255dc3f5fd9de (DEFAULT_TABLE table=test-table [id=99f3a4913be54af08ba9edf3a7f98bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:41.197359 29836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809b0ade4abf4dc89db255dc3f5fd9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:41.197371 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809b0ade4abf4dc89db255dc3f5fd9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:41.197777 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.192245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:38323 (local address 127.0.0.1:48460)
0925 02:30:41.192513 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.192522 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.192561 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.193095 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.193095 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.193097 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:41.193132 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.193133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.194355 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.194356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.195942 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.195943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.197578 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.197579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.197638 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.197642 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.197671 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.197709 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":83,"threads_started":1}
I20260925 02:30:41.198110 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.192759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.11.220.193:38323)
0925 02:30:41.192933 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.192937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.192946 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:30:41.192986 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.192988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.193002 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.193197 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:41.193269 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.194258 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.196716 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.197422 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.197901 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.197964 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.197990 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.198034 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260925 02:30:41.199862 29917 tablet_bootstrap.cc:492] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: Bootstrap starting.
I20260925 02:30:41.200013 29918 tablet_bootstrap.cc:492] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: Bootstrap starting.
I20260925 02:30:41.200740 29917 tablet_bootstrap.cc:654] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:41.200757 29918 tablet_bootstrap.cc:654] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:41.200668 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 809b0ade4abf4dc89db255dc3f5fd9de (DEFAULT_TABLE table=test-table [id=99f3a4913be54af08ba9edf3a7f98bab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:41.200973 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809b0ade4abf4dc89db255dc3f5fd9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:41.201265 29917 log.cc:824] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:41.201297 29918 log.cc:824] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:41.201928 29917 tablet_bootstrap.cc:492] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: No bootstrap required, opened a new log
I20260925 02:30:41.202008 29917 ts_tablet_manager.cc:1397] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 02:30:41.203423 29917 raft_consensus.cc:348] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 } } peers { permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 } } peers { permanent_uuid: "5cdc9455d2be4e099c518867f638a978" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40451 } }
I20260925 02:30:41.203573 29917 raft_consensus.cc:374] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:41.203609 29917 raft_consensus.cc:729] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdc9455d2be4e099c518867f638a978, State: Initialized, Role: FOLLOWER
I20260925 02:30:41.203640 29921 tablet_bootstrap.cc:492] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79: Bootstrap starting.
I20260925 02:30:41.203707 29917 consensus_queue.cc:260] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 } } peers { permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 } } peers { permanent_uuid: "5cdc9455d2be4e099c518867f638a978" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40451 } }
I20260925 02:30:41.203971 29917 ts_tablet_manager.cc:1428] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:30:41.203987 29903 heartbeater.cc:503] Master 127.11.220.254:35113 was elected leader, sending a full tablet report...
I20260925 02:30:41.204227 29921 tablet_bootstrap.cc:654] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:41.204425 29918 tablet_bootstrap.cc:492] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: No bootstrap required, opened a new log
I20260925 02:30:41.204533 29918 ts_tablet_manager.cc:1397] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260925 02:30:41.204839 29921 log.cc:824] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:41.205680 29921 tablet_bootstrap.cc:492] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79: No bootstrap required, opened a new log
I20260925 02:30:41.205730 29921 ts_tablet_manager.cc:1397] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:30:41.206212 29918 raft_consensus.cc:348] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 } } peers { permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 } } peers { permanent_uuid: "5cdc9455d2be4e099c518867f638a978" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40451 } }
I20260925 02:30:41.206379 29918 raft_consensus.cc:374] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:41.206421 29918 raft_consensus.cc:729] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edcaf0393ee7488bb00348496e711500, State: Initialized, Role: FOLLOWER
I20260925 02:30:41.206525 29918 consensus_queue.cc:260] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 } } peers { permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 } } peers { permanent_uuid: "5cdc9455d2be4e099c518867f638a978" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40451 } }
I20260925 02:30:41.206779 29918 ts_tablet_manager.cc:1428] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 02:30:41.206789 29767 heartbeater.cc:503] Master 127.11.220.254:35113 was elected leader, sending a full tablet report...
I20260925 02:30:41.207484 29921 raft_consensus.cc:348] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 } } peers { permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 } } peers { permanent_uuid: "5cdc9455d2be4e099c518867f638a978" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40451 } }
I20260925 02:30:41.207628 29921 raft_consensus.cc:374] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:41.207667 29921 raft_consensus.cc:729] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1133a310c2a542a1a8013198c3f94b79, State: Initialized, Role: FOLLOWER
I20260925 02:30:41.207763 29921 consensus_queue.cc:260] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 } } peers { permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 } } peers { permanent_uuid: "5cdc9455d2be4e099c518867f638a978" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40451 } }
I20260925 02:30:41.207988 29921 ts_tablet_manager.cc:1428] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:30:41.208015 29630 heartbeater.cc:503] Master 127.11.220.254:35113 was elected leader, sending a full tablet report...
I20260925 02:30:41.231289 29922 raft_consensus.cc:482] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:41.231351 29922 raft_consensus.cc:504] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 } } peers { permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 } } peers { permanent_uuid: "5cdc9455d2be4e099c518867f638a978" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40451 } }
I20260925 02:30:41.231588 29922 leader_election.cc:288] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edcaf0393ee7488bb00348496e711500 (127.11.220.194:42691), 1133a310c2a542a1a8013198c3f94b79 (127.11.220.193:38323)
I20260925 02:30:41.234937 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.231776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:56759 (local address 127.11.220.193:38323)
0925 02:30:41.231816 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.231817 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.231940 (+ 123us) server_negotiation.cc:407] Connection header received
0925 02:30:41.231966 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.231967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.231975 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.232021 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:41.232175 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.232847 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.234040 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.234609 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.234676 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.234683 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.234839 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.234875 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 02:30:41.234937 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.231775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:42125 (local address 127.11.220.194:42691)
0925 02:30:41.231808 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.231809 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.231817 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:30:41.231854 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.231855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.231862 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.231948 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:41.232104 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.232810 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.234040 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.234602 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.234676 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.234683 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.234848 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.234878 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:30:41.234979 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.231671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:42691 (local address 127.11.220.195:42125)
0925 02:30:41.231719 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.231731 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.231768 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.231955 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.231956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.231957 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:41.232034 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.232036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.232872 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.232873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.233962 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.233964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.234685 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.234686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.234763 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.234765 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.234837 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.234861 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":6}
I20260925 02:30:41.235185 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.231741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:38323 (local address 127.11.220.195:56759)
0925 02:30:41.231869 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.231877 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.231922 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.232049 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.232050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.232051 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:41.232107 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.232108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.232908 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.232908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.233962 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.233964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.234685 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.234686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.234760 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.234764 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.234821 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.234859 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":6,"thread_start_us":64,"threads_started":1}
I20260925 02:30:41.235320 29721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809b0ade4abf4dc89db255dc3f5fd9de" candidate_uuid: "5cdc9455d2be4e099c518867f638a978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edcaf0393ee7488bb00348496e711500" is_pre_election: true
I20260925 02:30:41.235445 29721 raft_consensus.cc:2454] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cdc9455d2be4e099c518867f638a978 in term 0.
I20260925 02:30:41.235466 29582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809b0ade4abf4dc89db255dc3f5fd9de" candidate_uuid: "5cdc9455d2be4e099c518867f638a978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1133a310c2a542a1a8013198c3f94b79" is_pre_election: true
I20260925 02:30:41.235591 29582 raft_consensus.cc:2454] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cdc9455d2be4e099c518867f638a978 in term 0.
I20260925 02:30:41.235687 29789 leader_election.cc:302] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cdc9455d2be4e099c518867f638a978, edcaf0393ee7488bb00348496e711500; no voters:
I20260925 02:30:41.235796 29922 raft_consensus.cc:2781] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:30:41.235886 29922 raft_consensus.cc:482] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:30:41.235919 29922 raft_consensus.cc:3037] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:41.236509 29922 raft_consensus.cc:504] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 } } peers { permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 } } peers { permanent_uuid: "5cdc9455d2be4e099c518867f638a978" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40451 } }
I20260925 02:30:41.236636 29922 leader_election.cc:288] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [CANDIDATE]: Term 1 election: Requested vote from peers edcaf0393ee7488bb00348496e711500 (127.11.220.194:42691), 1133a310c2a542a1a8013198c3f94b79 (127.11.220.193:38323)
I20260925 02:30:41.236760 29721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809b0ade4abf4dc89db255dc3f5fd9de" candidate_uuid: "5cdc9455d2be4e099c518867f638a978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edcaf0393ee7488bb00348496e711500"
I20260925 02:30:41.236776 29582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809b0ade4abf4dc89db255dc3f5fd9de" candidate_uuid: "5cdc9455d2be4e099c518867f638a978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1133a310c2a542a1a8013198c3f94b79"
I20260925 02:30:41.236843 29721 raft_consensus.cc:3037] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:41.236842 29582 raft_consensus.cc:3037] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:41.237406 29721 raft_consensus.cc:2454] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdc9455d2be4e099c518867f638a978 in term 1.
I20260925 02:30:41.237437 29582 raft_consensus.cc:2454] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdc9455d2be4e099c518867f638a978 in term 1.
I20260925 02:30:41.237578 29789 leader_election.cc:302] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cdc9455d2be4e099c518867f638a978, edcaf0393ee7488bb00348496e711500; no voters:
I20260925 02:30:41.237649 29922 raft_consensus.cc:2781] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:41.237751 29922 raft_consensus.cc:686] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 1 LEADER]: Becoming Leader. State: Replica: 5cdc9455d2be4e099c518867f638a978, State: Running, Role: LEADER
I20260925 02:30:41.237828 29922 consensus_queue.cc:237] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 } } peers { permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 } } peers { permanent_uuid: "5cdc9455d2be4e099c518867f638a978" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40451 } }
I20260925 02:30:41.238449 29433 catalog_manager.cc:5495] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cdc9455d2be4e099c518867f638a978 (127.11.220.195). New cstate: current_term: 1 leader_uuid: "5cdc9455d2be4e099c518867f638a978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cdc9455d2be4e099c518867f638a978" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 40451 } health_report { overall_health: HEALTHY } } }
I20260925 02:30:41.243283 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.241436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.11.220.195:40451)
0925 02:30:41.241468 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.241469 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.241496 (+ 27us) server_negotiation.cc:407] Connection header received
0925 02:30:41.241564 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.241565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.241583 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.241630 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:30:41.241719 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.242105 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.242737 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.242885 (+ 148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.242926 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.243060 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:30:41.243132 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:30:41.243185 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.243218 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.243245 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:30:41.244539 29721 raft_consensus.cc:1261] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500 [term 1 FOLLOWER]: Refusing update from remote peer 5cdc9455d2be4e099c518867f638a978: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:41.244553 29582 raft_consensus.cc:1261] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79 [term 1 FOLLOWER]: Refusing update from remote peer 5cdc9455d2be4e099c518867f638a978: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:41.244817 29922 consensus_queue.cc:1035] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edcaf0393ee7488bb00348496e711500" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:41.244925 29927 consensus_queue.cc:1035] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1133a310c2a542a1a8013198c3f94b79" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:41.246870 29932 mvcc.cc:205] Tried to move back new op lower bound from 7333082895335763968 to 7333082895310671872. Current Snapshot: MvccSnapshot[applied={T|T < 7333082895335763968}]
I20260925 02:30:41.246980 29930 mvcc.cc:205] Tried to move back new op lower bound from 7333082895335763968 to 7333082895310671872. Current Snapshot: MvccSnapshot[applied={T|T < 7333082895335763968 or (T in {7333082895335763968})}]
I20260925 02:30:41.247064 29931 mvcc.cc:205] Tried to move back new op lower bound from 7333082895335763968 to 7333082895310671872. Current Snapshot: MvccSnapshot[applied={T|T < 7333082895335763968}]
I20260925 02:30:41.248507 29433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:41.248591 29433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:41.249174 29433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98e8c7b15e6d40fb96c48ce8e5045656: Sending DeleteTablet for 3 replicas of tablet 809b0ade4abf4dc89db255dc3f5fd9de
I20260925 02:30:41.249465 29562 tablet_service.cc:1514] Processing DeleteTablet for tablet 809b0ade4abf4dc89db255dc3f5fd9de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:41 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:48460
I20260925 02:30:41.249500 29700 tablet_service.cc:1514] Processing DeleteTablet for tablet 809b0ade4abf4dc89db255dc3f5fd9de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:41 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:36698
I20260925 02:30:41.249526 29836 tablet_service.cc:1514] Processing DeleteTablet for tablet 809b0ade4abf4dc89db255dc3f5fd9de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:41 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:35784
I20260925 02:30:41.249666 29939 tablet_replica.cc:330] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79: stopping tablet replica
I20260925 02:30:41.249752 29939 raft_consensus.cc:2229] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:41.249841 29939 raft_consensus.cc:2258] T 809b0ade4abf4dc89db255dc3f5fd9de P 1133a310c2a542a1a8013198c3f94b79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:41.249945 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 29497
I20260925 02:30:41.249985 29941 tablet_replica.cc:330] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: stopping tablet replica
I20260925 02:30:41.250061 29941 raft_consensus.cc:2229] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:30:41.250206 29940 tablet_replica.cc:330] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: stopping tablet replica
I20260925 02:30:41.250344 29940 raft_consensus.cc:2229] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:41.250433 29940 raft_consensus.cc:2258] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:41.250834 29940 ts_tablet_manager.cc:1905] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:41.251124 29941 raft_consensus.cc:2258] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:41.251574 29941 ts_tablet_manager.cc:1905] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:41.252441 29940 ts_tablet_manager.cc:1918] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:41.252552 29940 log.cc:1196] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/809b0ade4abf4dc89db255dc3f5fd9de
I20260925 02:30:41.252846 29940 ts_tablet_manager.cc:1939] T 809b0ade4abf4dc89db255dc3f5fd9de P edcaf0393ee7488bb00348496e711500: Deleting consensus metadata
I20260925 02:30:41.253299 29418 catalog_manager.cc:4841] TS edcaf0393ee7488bb00348496e711500 (127.11.220.194:42691): tablet 809b0ade4abf4dc89db255dc3f5fd9de (table test-table [id=99f3a4913be54af08ba9edf3a7f98bab]) successfully deleted
I20260925 02:30:41.253407 29941 ts_tablet_manager.cc:1918] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:41.253472 29941 log.cc:1196] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/809b0ade4abf4dc89db255dc3f5fd9de
I20260925 02:30:41.253710 29941 ts_tablet_manager.cc:1939] T 809b0ade4abf4dc89db255dc3f5fd9de P 5cdc9455d2be4e099c518867f638a978: Deleting consensus metadata
I20260925 02:30:41.254093 29418 catalog_manager.cc:4841] TS 5cdc9455d2be4e099c518867f638a978 (127.11.220.195:40451): tablet 809b0ade4abf4dc89db255dc3f5fd9de (table test-table [id=99f3a4913be54af08ba9edf3a7f98bab]) successfully deleted
W20260925 02:30:41.257625 29419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:38323 (error 108)
I20260925 02:30:41.258044 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.257930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:38323 (local address 127.0.0.1:48474)
0925 02:30:41.257978 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.258012 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:38323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 02:30:41.258111 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 29635
W20260925 02:30:41.258177 29419 catalog_manager.cc:4584] TS 1133a310c2a542a1a8013198c3f94b79 (127.11.220.193:38323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 809b0ade4abf4dc89db255dc3f5fd9de: Network error: Client connection negotiation failed: client connection to 127.11.220.193:38323: connect: Connection refused (error 111)
I20260925 02:30:41.264238 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 29772
I20260925 02:30:41.269007 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 29401
2026-09-25T02:30:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1752 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T02:30:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:41Z Disabled control of system clock
I20260925 02:30:41.294703 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:44041
--webserver_interface=127.11.220.254
--webserver_port=0
--builtin_ntp_servers=127.11.220.212:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:44041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:30:41.367079 29946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:41.367247 29946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:41.367265 29946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:41.368398 29946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:41.368435 29946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:41.368481 29946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:41.368505 29946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:41.368546 29946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:41.369911 29946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:44041
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:44041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:41.370146 29946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:41.370390 29946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:41.374464 29951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:41.374516 29952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:41.374634 29946 server_base.cc:1034] running on GCE node
W20260925 02:30:41.374545 29954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:41.374934 29946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:41.375155 29946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:41.376319 29946 hybrid_clock.cc:648] HybridClock initialized: now 1790303441376309 us; error 26 us; skew 500 ppm
I20260925 02:30:41.377396 29946 webserver.cc:460] Webserver started at http://127.11.220.254:36703/ using document root <none> and password file <none>
I20260925 02:30:41.377619 29946 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:41.377660 29946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:41.377790 29946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:41.378636 29946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "868986bcb8c048e4ba45fa415c423a6c"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "818e0e6bf2bb9e3caf1e7b3ffdee7e97"
server_key_iv: "dd839cfe6bcaeb064c40ca05cce29870"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.378942 29946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "868986bcb8c048e4ba45fa415c423a6c"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "818e0e6bf2bb9e3caf1e7b3ffdee7e97"
server_key_iv: "dd839cfe6bcaeb064c40ca05cce29870"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.380113 29946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:30:41.380880 29961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.381034 29946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.381095 29946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "868986bcb8c048e4ba45fa415c423a6c"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "818e0e6bf2bb9e3caf1e7b3ffdee7e97"
server_key_iv: "dd839cfe6bcaeb064c40ca05cce29870"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.381175 29946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:41.390690 29946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:41.390972 29946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:41.391116 29946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:41.394831 29946 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:44041
I20260925 02:30:41.394857 30014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:44041 every 8 connection(s)
I20260925 02:30:41.395155 29946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:41.395982 30015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:41.397861 30015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c: Bootstrap starting.
I20260925 02:30:41.398108 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 29946
I20260925 02:30:41.398178 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:41.398346 12147 external_mini_cluster.cc:1428] Setting key aba42441d891b41685345115d7c454bd
I20260925 02:30:41.398416 30015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:41.398726 30015 log.cc:824] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:41.399402 30015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c: No bootstrap required, opened a new log
I20260925 02:30:41.401652 30015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868986bcb8c048e4ba45fa415c423a6c" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44041 } }
I20260925 02:30:41.401767 30015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:41.401787 30015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868986bcb8c048e4ba45fa415c423a6c, State: Initialized, Role: FOLLOWER
I20260925 02:30:41.401926 30015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868986bcb8c048e4ba45fa415c423a6c" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44041 } }
I20260925 02:30:41.402020 30015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:41.402060 30015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:41.402117 30015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:41.402604 30015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868986bcb8c048e4ba45fa415c423a6c" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44041 } }
I20260925 02:30:41.402711 30015 leader_election.cc:302] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 868986bcb8c048e4ba45fa415c423a6c; no voters:
I20260925 02:30:41.402851 30015 leader_election.cc:288] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:41.402889 30020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:41.403038 30020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [term 1 LEADER]: Becoming Leader. State: Replica: 868986bcb8c048e4ba45fa415c423a6c, State: Running, Role: LEADER
I20260925 02:30:41.403100 30015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:41.403115 30020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868986bcb8c048e4ba45fa415c423a6c" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44041 } }
I20260925 02:30:41.403416 30021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "868986bcb8c048e4ba45fa415c423a6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868986bcb8c048e4ba45fa415c423a6c" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44041 } } }
I20260925 02:30:41.403479 30021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:41.403625 30022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 868986bcb8c048e4ba45fa415c423a6c. Latest consensus state: current_term: 1 leader_uuid: "868986bcb8c048e4ba45fa415c423a6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "868986bcb8c048e4ba45fa415c423a6c" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44041 } } }
I20260925 02:30:41.403679 30022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:41.403961 30026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:41.404177 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.398783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53628 (local address 127.11.220.254:44041)
0925 02:30:41.398967 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.398973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.398994 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:30:41.399444 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.399451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.399523 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.399662 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:41.399808 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.401140 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.401904 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.402373 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.402501 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.403184 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:41.403187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:41.403187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:41.403370 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:41.403405 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:41.403419 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:41.403454 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.403819 (+ 365us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.403892 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260925 02:30:41.404398 30026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:41.405722 30026 catalog_manager.cc:1347] Generated new cluster ID: 8b20105886a1485ab5aba880f7a0256c
I20260925 02:30:41.405774 30026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:41.417431 30026 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:41.418054 30026 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:41.428876 30026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c: Generated new TSK 0
I20260925 02:30:41.429028 30026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:41.434970 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44041
--builtin_ntp_servers=127.11.220.212:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:30:41.519248 30039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:41.519457 30039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:41.519495 30039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:41.520654 30039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:41.520689 30039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:41.520722 30039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:30:41.521963 30039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:41.522176 30039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:41.522449 30039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:41.527432 30044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:41.527446 30045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:41.527599 30039 server_base.cc:1034] running on GCE node
W20260925 02:30:41.527441 30047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:41.527828 30039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:41.528060 30039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:41.529209 30039 hybrid_clock.cc:648] HybridClock initialized: now 1790303441529198 us; error 26 us; skew 500 ppm
I20260925 02:30:41.530366 30039 webserver.cc:460] Webserver started at http://127.11.220.193:41759/ using document root <none> and password file <none>
I20260925 02:30:41.530563 30039 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:41.530620 30039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:41.530750 30039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:41.531613 30039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "07c02120d59f4f188ee5af25943a04f0"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "594e2ddfbb52401c8bdb28dce28865ea"
server_key_iv: "774823d490fbce0115c30b6407058c59"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.531925 30039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "07c02120d59f4f188ee5af25943a04f0"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "594e2ddfbb52401c8bdb28dce28865ea"
server_key_iv: "774823d490fbce0115c30b6407058c59"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.533162 30039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:30:41.533835 30053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.534050 30039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:41.534135 30039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "07c02120d59f4f188ee5af25943a04f0"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "594e2ddfbb52401c8bdb28dce28865ea"
server_key_iv: "774823d490fbce0115c30b6407058c59"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.534195 30039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:41.543687 30039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:41.543872 30039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:41.543956 30039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:41.544097 30039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:41.544297 30039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:41.544323 30039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.544340 30039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:41.544351 30039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.549299 30039 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:33025
I20260925 02:30:41.549363 30167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:33025 every 8 connection(s)
I20260925 02:30:41.549626 30039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:30:41.549752 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 30039
I20260925 02:30:41.549877 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:30:41.550056 12147 external_mini_cluster.cc:1428] Setting key 736407f591786a36a1f102f6c8a24fc0
I20260925 02:30:41.550860 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44041
--builtin_ntp_servers=127.11.220.212:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 02:30:41.554553 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.550325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:60537 (local address 127.11.220.254:44041)
0925 02:30:41.550368 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.550370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.550571 (+ 201us) server_negotiation.cc:407] Connection header received
0925 02:30:41.550976 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.550977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.550988 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.551035 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:41.551613 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.552292 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.553211 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.553506 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.553573 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.554302 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:41.554304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:41.554304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:41.554309 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:41.554324 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:41.554329 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:41.554385 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.554435 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.554458 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:30:41.554657 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.550290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:44041 (local address 127.11.220.193:60537)
0925 02:30:41.550456 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.550487 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.550886 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.551087 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.551090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.551099 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:41.551502 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.551533 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.552386 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.552387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.553132 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.553134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.553588 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.553589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.553689 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.554187 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:30:41.554187 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:41.554208 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:30:41.554227 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:41.554386 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:41.554389 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:41.554391 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.554433 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.554497 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260925 02:30:41.555277 30168 heartbeater.cc:347] Connected to a master server at 127.11.220.254:44041
I20260925 02:30:41.555346 30168 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:41.555498 30168 heartbeater.cc:511] Master 127.11.220.254:44041 requested a full tablet report, sending...
I20260925 02:30:41.556027 29978 ts_manager.cc:194] Registered new tserver with Master: 07c02120d59f4f188ee5af25943a04f0 (127.11.220.193:33025)
I20260925 02:30:41.556679 29978 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.193:60537
W20260925 02:30:41.624713 30171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:41.624907 30171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:41.624944 30171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:41.626051 30171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:41.626111 30171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:41.626153 30171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:30:41.627381 30171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:41.627604 30171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:41.627849 30171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:41.632242 30179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:41.632268 30176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:41.632270 30177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:41.632393 30171 server_base.cc:1034] running on GCE node
I20260925 02:30:41.632635 30171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:41.632839 30171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:41.633972 30171 hybrid_clock.cc:648] HybridClock initialized: now 1790303441633957 us; error 28 us; skew 500 ppm
I20260925 02:30:41.635140 30171 webserver.cc:460] Webserver started at http://127.11.220.194:43785/ using document root <none> and password file <none>
I20260925 02:30:41.635350 30171 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:41.635389 30171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:41.635521 30171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:41.636407 30171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "de82f556c14542289f2eddaf79611298"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "142acbad95a753a311c864e367acb23f"
server_key_iv: "10437b98967577a103510d3f9498355a"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.636817 30171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "de82f556c14542289f2eddaf79611298"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "142acbad95a753a311c864e367acb23f"
server_key_iv: "10437b98967577a103510d3f9498355a"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.638145 30171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:41.638793 30185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.638955 30171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:41.639032 30171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "de82f556c14542289f2eddaf79611298"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "142acbad95a753a311c864e367acb23f"
server_key_iv: "10437b98967577a103510d3f9498355a"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.639096 30171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:41.658010 30171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:41.658282 30171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:41.658428 30171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:41.658622 30171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:41.658905 30171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:41.658952 30171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.658978 30171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:41.659005 30171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.664520 30171 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:33655
I20260925 02:30:41.664614 30302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:33655 every 8 connection(s)
I20260925 02:30:41.664840 30171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:30:41.665495 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 30171
I20260925 02:30:41.665644 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:30:41.665890 12147 external_mini_cluster.cc:1428] Setting key 3e00e187bf8d79893be24ec94d869815
I20260925 02:30:41.666814 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44041
--builtin_ntp_servers=127.11.220.212:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 02:30:41.670310 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.665547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:57745 (local address 127.11.220.254:44041)
0925 02:30:41.665641 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.665643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.665874 (+ 231us) server_negotiation.cc:407] Connection header received
0925 02:30:41.666442 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.666443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.666455 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.666516 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:41.667196 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.667926 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.668966 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.669295 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.669371 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.670046 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:41.670047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:41.670047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:41.670052 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:41.670066 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:41.670070 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:41.670130 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.670177 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.670202 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 02:30:41.670403 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.665461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:44041 (local address 127.11.220.194:57745)
0925 02:30:41.665760 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.665799 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.666360 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.666589 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.666593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.666603 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:41.667112 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.667116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.668034 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.668035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.668892 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.668894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.669381 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.669381 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.669472 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.669926 (+ 454us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:30:41.669927 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:41.669949 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:30:41.669967 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:41.670132 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:41.670133 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:41.670135 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.670175 (+ 40us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.670237 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":73,"threads_started":1}
I20260925 02:30:41.670899 30303 heartbeater.cc:347] Connected to a master server at 127.11.220.254:44041
I20260925 02:30:41.670989 30303 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:41.671130 30303 heartbeater.cc:511] Master 127.11.220.254:44041 requested a full tablet report, sending...
I20260925 02:30:41.671454 29978 ts_manager.cc:194] Registered new tserver with Master: de82f556c14542289f2eddaf79611298 (127.11.220.194:33655)
I20260925 02:30:41.671797 29978 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.194:57745
W20260925 02:30:41.740029 30306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:41.740187 30306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:41.740207 30306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:41.741334 30306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:41.741381 30306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:41.741420 30306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:30:41.742820 30306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:41.743028 30306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:41.743294 30306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:41.747383 30311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:41.747390 30314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:41.747438 30312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:41.747543 30306 server_base.cc:1034] running on GCE node
I20260925 02:30:41.747750 30306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:41.747949 30306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:41.749075 30306 hybrid_clock.cc:648] HybridClock initialized: now 1790303441749056 us; error 34 us; skew 500 ppm
I20260925 02:30:41.750164 30306 webserver.cc:460] Webserver started at http://127.11.220.195:45217/ using document root <none> and password file <none>
I20260925 02:30:41.750371 30306 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:41.750412 30306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:41.750540 30306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:41.751405 30306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "e485d71fea9346fcaa08f7a7f5306db3"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "75e808d83e220109943ec11c4db385db"
server_key_iv: "30a25ff135da2a2e67f2853413c48be0"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.751724 30306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "e485d71fea9346fcaa08f7a7f5306db3"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "75e808d83e220109943ec11c4db385db"
server_key_iv: "30a25ff135da2a2e67f2853413c48be0"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.752882 30306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:30:41.753517 30320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.753680 30306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:41.753768 30306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "e485d71fea9346fcaa08f7a7f5306db3"
format_stamp: "Formatted at 2026-09-25 02:30:41 on dist-test-slave-dn4n"
server_key: "75e808d83e220109943ec11c4db385db"
server_key_iv: "30a25ff135da2a2e67f2853413c48be0"
server_key_version: "encryptionkey@0"
I20260925 02:30:41.753841 30306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:41.776171 30306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:41.776433 30306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:41.776623 30306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:41.776822 30306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:41.777091 30306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:41.777143 30306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.777176 30306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:41.777200 30306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:41.782617 30306 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:46617
I20260925 02:30:41.782671 30437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:46617 every 8 connection(s)
I20260925 02:30:41.782972 30306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:30:41.787233 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.783548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:36385 (local address 127.11.220.254:44041)
0925 02:30:41.783612 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.783614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.783840 (+ 226us) server_negotiation.cc:407] Connection header received
0925 02:30:41.784242 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.784243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.784252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.784314 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:41.784926 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.785387 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.786221 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.786395 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.786438 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.787030 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:41.787031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:41.787032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:41.787035 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:41.787046 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:41.787049 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:41.787092 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.787164 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.787181 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:30:41.787418 30440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.783538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:44041 (local address 127.11.220.195:36385)
0925 02:30:41.783729 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.783764 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.784169 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.784365 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.784367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.784375 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:41.784864 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.784869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.785466 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.785467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.786138 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.786140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.786461 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.786461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.786551 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.786922 (+ 371us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:30:41.786922 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:41.786944 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:30:41.786963 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:41.787104 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:41.787107 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:41.787110 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.787154 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.787221 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260925 02:30:41.787850 30438 heartbeater.cc:347] Connected to a master server at 127.11.220.254:44041
I20260925 02:30:41.787916 30438 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:41.788070 30438 heartbeater.cc:511] Master 127.11.220.254:44041 requested a full tablet report, sending...
I20260925 02:30:41.788412 29978 ts_manager.cc:194] Registered new tserver with Master: e485d71fea9346fcaa08f7a7f5306db3 (127.11.220.195:46617)
I20260925 02:30:41.788775 29978 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.195:36385
I20260925 02:30:41.792030 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 30306
I20260925 02:30:41.792117 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:30:41.792258 12147 external_mini_cluster.cc:1428] Setting key 5fc222f214082b23be14eb366799aff1
I20260925 02:30:41.793072 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:30:41.796288 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.794227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.11.220.254:44041)
0925 02:30:41.794291 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.794293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.794340 (+ 47us) server_negotiation.cc:407] Connection header received
0925 02:30:41.794388 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.794388 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.794395 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.794423 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:41.794537 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.794912 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.795505 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.795652 (+ 147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.795693 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.796118 (+ 425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:41.796119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:41.796119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:41.796122 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:41.796131 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:41.796134 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:41.796174 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.796206 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.796219 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:30:41.797547 29978 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:30:41.799007 29978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:30:41.804721 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.800972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:33655 (local address 127.0.0.1:58818)
0925 02:30:41.801185 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.801194 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.801244 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.801503 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.801505 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.801507 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:41.801564 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.801566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.802794 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.802795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.803550 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.803552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.804417 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.804418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.804530 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.804544 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.804608 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.804656 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":4,"thread_start_us":43,"threads_started":1}
I20260925 02:30:41.804850 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.800987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:46617 (local address 127.0.0.1:56988)
0925 02:30:41.801330 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.801338 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.801374 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.801575 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.801576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.801578 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:41.801619 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.801620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.802448 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.802449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.803666 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.803669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.804636 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.804637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.804690 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.804694 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.804736 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.804773 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260925 02:30:41.804930 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.801165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.11.220.194:33655)
0925 02:30:41.801348 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.801352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.801363 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:30:41.801399 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.801401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.801416 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.801500 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:41.801644 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.802713 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.803678 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.804330 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.804408 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.804508 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.804625 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.804657 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":75,"threads_started":1}
I20260925 02:30:41.805060 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.800972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:33025 (local address 127.0.0.1:38300)
0925 02:30:41.801138 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:41.801173 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:41.801227 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:41.801596 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:41.801596 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:41.801597 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:41.801622 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.801623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.802779 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.802780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.803812 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:41.803814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.804764 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.804765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:41.804901 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.804904 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:41.804951 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 02:30:41.804987 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"spinlock_wait_cycles":384,"thread_start_us":87,"threads_started":1}
I20260925 02:30:41.805155 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.801279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.11.220.195:46617)
0925 02:30:41.801406 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.801410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.801418 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:30:41.801446 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.801447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.801459 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.801519 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:41.801689 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.802372 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.803758 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.804448 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.804530 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.804602 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.804996 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.805050 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
I20260925 02:30:41.805347 30449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:41.801084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.11.220.193:33025)
0925 02:30:41.801455 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:41.801458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:41.801468 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:30:41.801505 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:41.801513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:41.801527 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:41.801602 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:41.801738 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.802699 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.803892 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:41.804609 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:41.804667 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:41.804737 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:41.805229 (+ 492us) server_negotiation.cc:299] Negotiation successful
0925 02:30:41.805272 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":79,"threads_started":1}
I20260925 02:30:41.805548 30235 tablet_service.cc:1467] Processing CreateTablet for tablet b030039528a54c65bd519ac96a21cc15 (DEFAULT_TABLE table=test-table [id=8c7f36c51a1d4cfc8fd58821168d3ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:41.805660 30371 tablet_service.cc:1467] Processing CreateTablet for tablet b030039528a54c65bd519ac96a21cc15 (DEFAULT_TABLE table=test-table [id=8c7f36c51a1d4cfc8fd58821168d3ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:41.805903 30235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b030039528a54c65bd519ac96a21cc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:41.805953 30371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b030039528a54c65bd519ac96a21cc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:41.805853 30101 tablet_service.cc:1467] Processing CreateTablet for tablet b030039528a54c65bd519ac96a21cc15 (DEFAULT_TABLE table=test-table [id=8c7f36c51a1d4cfc8fd58821168d3ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:41.806154 30101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b030039528a54c65bd519ac96a21cc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:41.808385 30452 tablet_bootstrap.cc:492] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0: Bootstrap starting.
I20260925 02:30:41.808986 30453 tablet_bootstrap.cc:492] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: Bootstrap starting.
I20260925 02:30:41.809187 30452 tablet_bootstrap.cc:654] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:41.809530 30452 log.cc:824] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:41.809625 30453 tablet_bootstrap.cc:654] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:41.809911 30454 tablet_bootstrap.cc:492] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: Bootstrap starting.
I20260925 02:30:41.809962 30453 log.cc:824] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:41.810369 30452 tablet_bootstrap.cc:492] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0: No bootstrap required, opened a new log
I20260925 02:30:41.810436 30452 ts_tablet_manager.cc:1397] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:41.810523 30454 tablet_bootstrap.cc:654] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:41.810681 30453 tablet_bootstrap.cc:492] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: No bootstrap required, opened a new log
I20260925 02:30:41.810748 30453 ts_tablet_manager.cc:1397] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:41.810824 30454 log.cc:824] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:41.811630 30454 tablet_bootstrap.cc:492] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: No bootstrap required, opened a new log
I20260925 02:30:41.811692 30454 ts_tablet_manager.cc:1397] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:41.812685 30452 raft_consensus.cc:348] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c02120d59f4f188ee5af25943a04f0" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33025 } } peers { permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 } } peers { permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 } }
I20260925 02:30:41.812829 30452 raft_consensus.cc:374] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:41.812870 30452 raft_consensus.cc:729] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c02120d59f4f188ee5af25943a04f0, State: Initialized, Role: FOLLOWER
I20260925 02:30:41.812983 30452 consensus_queue.cc:260] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c02120d59f4f188ee5af25943a04f0" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33025 } } peers { permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 } } peers { permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 } }
I20260925 02:30:41.813213 30452 ts_tablet_manager.cc:1428] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260925 02:30:41.813189 30453 raft_consensus.cc:348] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c02120d59f4f188ee5af25943a04f0" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33025 } } peers { permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 } } peers { permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 } }
I20260925 02:30:41.813329 30453 raft_consensus.cc:374] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:41.813338 30168 heartbeater.cc:503] Master 127.11.220.254:44041 was elected leader, sending a full tablet report...
I20260925 02:30:41.813364 30453 raft_consensus.cc:729] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e485d71fea9346fcaa08f7a7f5306db3, State: Initialized, Role: FOLLOWER
I20260925 02:30:41.813467 30453 consensus_queue.cc:260] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c02120d59f4f188ee5af25943a04f0" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33025 } } peers { permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 } } peers { permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 } }
I20260925 02:30:41.813691 30453 ts_tablet_manager.cc:1428] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:30:41.813892 30438 heartbeater.cc:503] Master 127.11.220.254:44041 was elected leader, sending a full tablet report...
I20260925 02:30:41.814116 30454 raft_consensus.cc:348] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c02120d59f4f188ee5af25943a04f0" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33025 } } peers { permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 } } peers { permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 } }
I20260925 02:30:41.814255 30454 raft_consensus.cc:374] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:41.814306 30454 raft_consensus.cc:729] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de82f556c14542289f2eddaf79611298, State: Initialized, Role: FOLLOWER
I20260925 02:30:41.814415 30454 consensus_queue.cc:260] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c02120d59f4f188ee5af25943a04f0" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33025 } } peers { permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 } } peers { permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 } }
I20260925 02:30:41.814671 30454 ts_tablet_manager.cc:1428] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:30:41.814703 30303 heartbeater.cc:503] Master 127.11.220.254:44041 was elected leader, sending a full tablet report...
W20260925 02:30:41.915403 30304 tablet.cc:2369] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:30:42.023033 30458 raft_consensus.cc:482] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:42.023159 30458 raft_consensus.cc:504] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c02120d59f4f188ee5af25943a04f0" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33025 } } peers { permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 } } peers { permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 } }
I20260925 02:30:42.023562 30458 leader_election.cc:288] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de82f556c14542289f2eddaf79611298 (127.11.220.194:33655), e485d71fea9346fcaa08f7a7f5306db3 (127.11.220.195:46617)
I20260925 02:30:42.026304 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:42.023821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:34675 (local address 127.11.220.195:46617)
0925 02:30:42.023863 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:42.023865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:42.023991 (+ 126us) server_negotiation.cc:407] Connection header received
0925 02:30:42.024026 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:42.024027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:42.024037 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:42.024102 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:42.024268 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:42.024781 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:42.025676 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:42.025989 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:42.026037 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:42.026063 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:42.026196 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 02:30:42.026237 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 02:30:42.026320 30461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:42.023753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:46617 (local address 127.11.220.193:34675)
0925 02:30:42.023920 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:42.023929 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:42.023981 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:42.024105 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:42.024106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:42.024107 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:42.024204 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:42.024205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:42.024854 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:42.024855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:42.025563 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:42.025565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:42.026058 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:42.026059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:42.026131 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:42.026134 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:42.026172 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 02:30:42.026199 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260925 02:30:42.026343 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:42.023800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:33455 (local address 127.11.220.194:33655)
0925 02:30:42.023848 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:42.023850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:42.023859 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:30:42.023895 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:42.023895 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:42.023905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:42.023965 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:42.024177 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:42.024727 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:42.025669 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:42.025989 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:42.026072 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:42.026079 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:42.026244 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 02:30:42.026283 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 02:30:42.026333 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:42.023725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:33655 (local address 127.11.220.193:33455)
0925 02:30:42.023782 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:42.023792 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:42.023834 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:42.024010 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:42.024011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:42.024013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:42.024092 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:42.024094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:42.024797 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:42.024798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:42.025552 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:42.025554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:42.026089 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:42.026090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:42.026152 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:42.026155 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:42.026223 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 02:30:42.026255 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":8}
I20260925 02:30:42.026662 30391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b030039528a54c65bd519ac96a21cc15" candidate_uuid: "07c02120d59f4f188ee5af25943a04f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e485d71fea9346fcaa08f7a7f5306db3" is_pre_election: true
I20260925 02:30:42.026662 30255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b030039528a54c65bd519ac96a21cc15" candidate_uuid: "07c02120d59f4f188ee5af25943a04f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de82f556c14542289f2eddaf79611298" is_pre_election: true
I20260925 02:30:42.026795 30391 raft_consensus.cc:2454] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07c02120d59f4f188ee5af25943a04f0 in term 0.
I20260925 02:30:42.026795 30255 raft_consensus.cc:2454] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07c02120d59f4f188ee5af25943a04f0 in term 0.
I20260925 02:30:42.027007 30057 leader_election.cc:302] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07c02120d59f4f188ee5af25943a04f0, e485d71fea9346fcaa08f7a7f5306db3; no voters:
I20260925 02:30:42.027141 30458 raft_consensus.cc:2781] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:30:42.027208 30458 raft_consensus.cc:482] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:30:42.027233 30458 raft_consensus.cc:3037] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:42.027845 30458 raft_consensus.cc:504] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c02120d59f4f188ee5af25943a04f0" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33025 } } peers { permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 } } peers { permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 } }
I20260925 02:30:42.028043 30458 leader_election.cc:288] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [CANDIDATE]: Term 1 election: Requested vote from peers de82f556c14542289f2eddaf79611298 (127.11.220.194:33655), e485d71fea9346fcaa08f7a7f5306db3 (127.11.220.195:46617)
I20260925 02:30:42.028118 30255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b030039528a54c65bd519ac96a21cc15" candidate_uuid: "07c02120d59f4f188ee5af25943a04f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de82f556c14542289f2eddaf79611298"
I20260925 02:30:42.028112 30391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b030039528a54c65bd519ac96a21cc15" candidate_uuid: "07c02120d59f4f188ee5af25943a04f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e485d71fea9346fcaa08f7a7f5306db3"
I20260925 02:30:42.028185 30255 raft_consensus.cc:3037] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:42.028185 30391 raft_consensus.cc:3037] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:42.029139 30255 raft_consensus.cc:2454] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07c02120d59f4f188ee5af25943a04f0 in term 1.
I20260925 02:30:42.029139 30391 raft_consensus.cc:2454] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07c02120d59f4f188ee5af25943a04f0 in term 1.
I20260925 02:30:42.029315 30057 leader_election.cc:302] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07c02120d59f4f188ee5af25943a04f0, e485d71fea9346fcaa08f7a7f5306db3; no voters:
I20260925 02:30:42.029491 30458 raft_consensus.cc:2781] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:42.029634 30458 raft_consensus.cc:686] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 1 LEADER]: Becoming Leader. State: Replica: 07c02120d59f4f188ee5af25943a04f0, State: Running, Role: LEADER
I20260925 02:30:42.029735 30458 consensus_queue.cc:237] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c02120d59f4f188ee5af25943a04f0" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33025 } } peers { permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 } } peers { permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 } }
I20260925 02:30:42.030601 29977 catalog_manager.cc:5495] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07c02120d59f4f188ee5af25943a04f0 (127.11.220.193). New cstate: current_term: 1 leader_uuid: "07c02120d59f4f188ee5af25943a04f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c02120d59f4f188ee5af25943a04f0" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 33025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 } health_report { overall_health: UNKNOWN } } }
W20260925 02:30:42.033406 30439 tablet.cc:2369] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:30:42.050343 30169 tablet.cc:2369] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:30:42.087852 30449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:42.085973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.11.220.193:33025)
0925 02:30:42.086020 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:42.086022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:42.086029 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:30:42.086065 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:42.086065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:42.086082 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:42.086134 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:30:42.086234 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:42.086644 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:42.087248 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:42.087405 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:42.087448 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:42.087561 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:30:42.087659 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:30:42.087710 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:42.087755 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 02:30:42.087776 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 02:30:42.089371 30391 raft_consensus.cc:1261] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3 [term 1 FOLLOWER]: Refusing update from remote peer 07c02120d59f4f188ee5af25943a04f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:42.089401 30255 raft_consensus.cc:1261] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298 [term 1 FOLLOWER]: Refusing update from remote peer 07c02120d59f4f188ee5af25943a04f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:42.089679 30462 consensus_queue.cc:1035] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e485d71fea9346fcaa08f7a7f5306db3" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 46617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:42.089794 30458 consensus_queue.cc:1035] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de82f556c14542289f2eddaf79611298" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 33655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:42.092167 30466 mvcc.cc:205] Tried to move back new op lower bound from 7333082898795741184 to 7333082898554585088. Current Snapshot: MvccSnapshot[applied={T|T < 7333082898795741184 or (T in {7333082898795741184})}]
I20260925 02:30:42.092622 30465 mvcc.cc:205] Tried to move back new op lower bound from 7333082898795741184 to 7333082898554585088. Current Snapshot: MvccSnapshot[applied={T|T < 7333082898795741184 or (T in {7333082898795741184})}]
I20260925 02:30:42.093286 30467 mvcc.cc:205] Tried to move back new op lower bound from 7333082898795741184 to 7333082898554585088. Current Snapshot: MvccSnapshot[applied={T|T < 7333082898795741184 or (T in {7333082898795741184})}]
I20260925 02:30:42.093734 29978 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:42.093814 29978 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:42.094499 29978 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 868986bcb8c048e4ba45fa415c423a6c: Sending DeleteTablet for 3 replicas of tablet b030039528a54c65bd519ac96a21cc15
I20260925 02:30:42.094795 30101 tablet_service.cc:1514] Processing DeleteTablet for tablet b030039528a54c65bd519ac96a21cc15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:42 UTC) from {username='slave'} at 127.0.0.1:38300
I20260925 02:30:42.094854 30371 tablet_service.cc:1514] Processing DeleteTablet for tablet b030039528a54c65bd519ac96a21cc15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:42 UTC) from {username='slave'} at 127.0.0.1:56988
I20260925 02:30:42.095058 30475 tablet_replica.cc:330] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0: stopping tablet replica
I20260925 02:30:42.095041 30235 tablet_service.cc:1514] Processing DeleteTablet for tablet b030039528a54c65bd519ac96a21cc15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:42 UTC) from {username='slave'} at 127.0.0.1:58818
I20260925 02:30:42.095144 30475 raft_consensus.cc:2229] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:30:42.095212 30475 raft_consensus.cc:2258] T b030039528a54c65bd519ac96a21cc15 P 07c02120d59f4f188ee5af25943a04f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:42.095227 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 30039
I20260925 02:30:42.095923 30477 tablet_replica.cc:330] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: stopping tablet replica
I20260925 02:30:42.096019 30477 raft_consensus.cc:2229] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:42.096110 30477 raft_consensus.cc:2258] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:42.096257 30476 tablet_replica.cc:330] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: stopping tablet replica
I20260925 02:30:42.096360 30476 raft_consensus.cc:2229] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:42.096455 30476 raft_consensus.cc:2258] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:42.096903 30477 ts_tablet_manager.cc:1905] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:42.098320 30476 ts_tablet_manager.cc:1905] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:42.098384 30477 ts_tablet_manager.cc:1918] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:42.098438 30477 log.cc:1196] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/b030039528a54c65bd519ac96a21cc15
I20260925 02:30:42.098706 30477 ts_tablet_manager.cc:1939] T b030039528a54c65bd519ac96a21cc15 P de82f556c14542289f2eddaf79611298: Deleting consensus metadata
I20260925 02:30:42.099100 29964 catalog_manager.cc:4841] TS de82f556c14542289f2eddaf79611298 (127.11.220.194:33655): tablet b030039528a54c65bd519ac96a21cc15 (table test-table [id=8c7f36c51a1d4cfc8fd58821168d3ed8]) successfully deleted
I20260925 02:30:42.099797 30476 ts_tablet_manager.cc:1918] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:42.099848 30476 log.cc:1196] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/b030039528a54c65bd519ac96a21cc15
I20260925 02:30:42.100075 30476 ts_tablet_manager.cc:1939] T b030039528a54c65bd519ac96a21cc15 P e485d71fea9346fcaa08f7a7f5306db3: Deleting consensus metadata
I20260925 02:30:42.100431 29965 catalog_manager.cc:4841] TS e485d71fea9346fcaa08f7a7f5306db3 (127.11.220.195:46617): tablet b030039528a54c65bd519ac96a21cc15 (table test-table [id=8c7f36c51a1d4cfc8fd58821168d3ed8]) successfully deleted
W20260925 02:30:42.100885 29963 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:33025 (error 108)
I20260925 02:30:42.101037 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 30171
I20260925 02:30:42.101603 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:42.101189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:33025 (local address 127.0.0.1:38322)
0925 02:30:42.101544 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:42.101575 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:33025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340}
W20260925 02:30:42.101714 29963 catalog_manager.cc:4584] TS 07c02120d59f4f188ee5af25943a04f0 (127.11.220.193:33025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b030039528a54c65bd519ac96a21cc15: Network error: Client connection negotiation failed: client connection to 127.11.220.193:33025: connect: Connection refused (error 111)
I20260925 02:30:42.106401 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 30306
I20260925 02:30:42.110992 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 29946
2026-09-25T02:30:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (841 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:42 dist-test-slave-dn4n krb5kdc[30481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:42 dist-test-slave-dn4n krb5kdc[30481](info): set up 2 sockets
Sep 25 02:30:42 dist-test-slave-dn4n krb5kdc[30481](info): commencing operation
krb5kdc: starting...
W20260925 02:30:43.065078 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:43 dist-test-slave-dn4n krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:30:43.165035 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:37207
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:33085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:37207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:43.237879 30494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:43.238049 30494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:43.238070 30494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:43.239022 30494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:43.239051 30494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:43.239063 30494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:43.239075 30494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:43.239087 30494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:43.240330 30494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:33085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:37207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:37207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:43.240548 30494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:43.240703 30494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:43.244953 30502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:43.244989 30494 server_base.cc:1034] running on GCE node
W20260925 02:30:43.244961 30499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:43.244975 30500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:43.245397 30494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:43.245671 30494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:43.246826 30494 hybrid_clock.cc:648] HybridClock initialized: now 1790303443246822 us; error 34 us; skew 500 ppm
Sep 25 02:30:43 dist-test-slave-dn4n krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303443, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:43.249039 30494 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:30:43.249423 30494 webserver.cc:460] Webserver started at http://127.11.220.254:42277/ using document root <none> and password file <none>
I20260925 02:30:43.249588 30494 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:43.249622 30494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:43.249697 30494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:43.250515 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "229ba3f94ff04524b0fd2d2a4313a3dd"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "9a4f87d0bf8bff0c55c40a0066826e15"
server_key_iv: "cb050ccc8c6e0db083533e3d3df09055"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.250770 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "229ba3f94ff04524b0fd2d2a4313a3dd"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "9a4f87d0bf8bff0c55c40a0066826e15"
server_key_iv: "cb050ccc8c6e0db083533e3d3df09055"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.251873 30494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:30:43.252532 30509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.252714 30494 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.252795 30494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "229ba3f94ff04524b0fd2d2a4313a3dd"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "9a4f87d0bf8bff0c55c40a0066826e15"
server_key_iv: "cb050ccc8c6e0db083533e3d3df09055"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.252866 30494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:43.273036 30494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:43.275167 30494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:43.275326 30494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:43.278882 30494 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:37207
I20260925 02:30:43.278924 30562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:37207 every 8 connection(s)
I20260925 02:30:43.279237 30494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:43.280053 30563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:43.282223 30563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd: Bootstrap starting.
I20260925 02:30:43.282756 30563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:43.283036 30563 log.cc:824] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:43.283651 30563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd: No bootstrap required, opened a new log
I20260925 02:30:43.285254 30563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229ba3f94ff04524b0fd2d2a4313a3dd" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37207 } }
I20260925 02:30:43.285387 30563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:43.285444 30563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229ba3f94ff04524b0fd2d2a4313a3dd, State: Initialized, Role: FOLLOWER
I20260925 02:30:43.285588 30563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229ba3f94ff04524b0fd2d2a4313a3dd" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37207 } }
I20260925 02:30:43.285660 30563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:43.285697 30563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:43.285742 30563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:43.286238 30563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229ba3f94ff04524b0fd2d2a4313a3dd" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37207 } }
I20260925 02:30:43.286351 30563 leader_election.cc:302] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 229ba3f94ff04524b0fd2d2a4313a3dd; no voters:
I20260925 02:30:43.286520 30563 leader_election.cc:288] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:43.286547 30566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:43.286671 30566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [term 1 LEADER]: Becoming Leader. State: Replica: 229ba3f94ff04524b0fd2d2a4313a3dd, State: Running, Role: LEADER
I20260925 02:30:43.286743 30563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:43.286813 30566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229ba3f94ff04524b0fd2d2a4313a3dd" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37207 } }
I20260925 02:30:43.287146 30569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 229ba3f94ff04524b0fd2d2a4313a3dd. Latest consensus state: current_term: 1 leader_uuid: "229ba3f94ff04524b0fd2d2a4313a3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229ba3f94ff04524b0fd2d2a4313a3dd" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37207 } } }
I20260925 02:30:43.287252 30568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "229ba3f94ff04524b0fd2d2a4313a3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229ba3f94ff04524b0fd2d2a4313a3dd" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37207 } } }
I20260925 02:30:43.287288 30569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:43.287325 30568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:43.287660 30576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:43.288087 30576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:43.289052 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 30494
I20260925 02:30:43.289165 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:43.289335 12147 external_mini_cluster.cc:1428] Setting key b065adfa95a1d5267fee202a4ca8443f
I20260925 02:30:43.289955 30576 catalog_manager.cc:1347] Generated new cluster ID: 809fd74972fb4159a380db71d8aa4ba4
I20260925 02:30:43.290001 30576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:30:43 dist-test-slave-dn4n krb5kdc[30481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:43.299783 30587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.289745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.11.220.254:37207)
0925 02:30:43.290040 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:43.290048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:43.290085 (+ 37us) server_negotiation.cc:407] Connection header received
0925 02:30:43.290207 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:43.290214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:43.290263 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:43.290386 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:43.291138 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.292510 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.293513 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.294052 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.294136 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.296594 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:43.296597 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:43.296597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:43.296607 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:43.298336 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:43.298674 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:43.298676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:43.298677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:43.298769 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:43.298955 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:43.298956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:43.298957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:43.299213 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:43.299357 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:43.299437 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 02:30:43.299579 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":83,"threads_started":1}
I20260925 02:30:43.305590 30576 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:43.306219 30576 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:43.312191 30576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd: Generated new TSK 0
I20260925 02:30:43.312330 30576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:30:43.336294 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37207
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:33085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:43.409843 30590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:43.410013 30590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:43.410035 30590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:43.411125 30590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:43.411162 30590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:43.411187 30590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:30:43.412505 30590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:33085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:43.412706 30590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:43.412917 30590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:43.417476 30598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:43.417570 30595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:43.417500 30596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:43.417601 30590 server_base.cc:1034] running on GCE node
I20260925 02:30:43.417824 30590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:43.418076 30590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:43.419231 30590 hybrid_clock.cc:648] HybridClock initialized: now 1790303443419226 us; error 37 us; skew 500 ppm
Sep 25 02:30:43 dist-test-slave-dn4n krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303443, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:43.421573 30590 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:30:43.421963 30590 webserver.cc:460] Webserver started at http://127.11.220.193:39343/ using document root <none> and password file <none>
I20260925 02:30:43.422173 30590 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:43.422214 30590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:43.422361 30590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:43.423262 30590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "424783d99b02900700250ada8e756eb6"
server_key_iv: "be815dfb47c7e2e3cd8921f427a65c7f"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.423564 30590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "424783d99b02900700250ada8e756eb6"
server_key_iv: "be815dfb47c7e2e3cd8921f427a65c7f"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.424757 30590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:30:43.425439 30605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.425591 30590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:43.425688 30590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "424783d99b02900700250ada8e756eb6"
server_key_iv: "be815dfb47c7e2e3cd8921f427a65c7f"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.425781 30590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:43.436812 30590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:43.438370 30590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:43.438514 30590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:43.438727 30590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:43.438977 30590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:43.439030 30590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.439065 30590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:43.439090 30590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.444425 30590 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:46501
I20260925 02:30:43.444516 30720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:46501 every 8 connection(s)
I20260925 02:30:43.444790 30590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
Sep 25 02:30:43 dist-test-slave-dn4n krb5kdc[30481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303443, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:43.449994 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 30590
I20260925 02:30:43.450119 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:30:43.450304 12147 external_mini_cluster.cc:1428] Setting key 686da9f3b128ba2d2a0f20f0a45f449c
I20260925 02:30:43.453207 30587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.445413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:49455 (local address 127.11.220.254:37207)
0925 02:30:43.445471 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:43.445472 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:43.445643 (+ 171us) server_negotiation.cc:407] Connection header received
0925 02:30:43.446024 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:43.446025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:43.446037 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:43.446088 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:43.446890 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.447353 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.448217 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.448382 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.448425 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.450511 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:43.450512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:43.450512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:43.450518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:43.451800 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:43.452278 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:43.452279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:43.452280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:43.452330 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:43.452643 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:43.452644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:43.452645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:43.452740 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:43.452792 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:43.453090 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 02:30:43.453149 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 02:30:43.453473 30723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.445370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:37207 (local address 127.11.220.193:49455)
0925 02:30:43.445518 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:43.445554 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:43.445949 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:43.446144 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:43.446147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:43.446643 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:43.446819 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.446822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.447427 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.447427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.448152 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.448153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.448459 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.448460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.448542 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.448983 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:43.448990 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:43.450406 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:43.451881 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:43.451888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:43.451890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:43.452201 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:43.452397 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:43.452398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:43.452398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:43.452553 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:43.452802 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:43.452809 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:43.452961 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 02:30:43.453071 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 02:30:43.453213 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260925 02:30:43.453918 30721 heartbeater.cc:347] Connected to a master server at 127.11.220.254:37207
I20260925 02:30:43.453998 30721 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:43.454149 30721 heartbeater.cc:511] Master 127.11.220.254:37207 requested a full tablet report, sending...
I20260925 02:30:43.454658 30526 ts_manager.cc:194] Registered new tserver with Master: 4800a1c6175e4e04a6c6e1b3d8d03af6 (127.11.220.193:46501)
I20260925 02:30:43.455312 30526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:49455
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:30:43.472832 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37207
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:33085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:43.543603 30726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:43.543776 30726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:43.543810 30726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:43.544780 30726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:43.544833 30726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:43.544871 30726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:30:43.546103 30726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:33085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:43.546314 30726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:43.546495 30726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:43.550796 30732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:43.550784 30734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:43.550822 30726 server_base.cc:1034] running on GCE node
W20260925 02:30:43.550771 30731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:43.551127 30726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:43.551285 30726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:43.552415 30726 hybrid_clock.cc:648] HybridClock initialized: now 1790303443552406 us; error 25 us; skew 500 ppm
Sep 25 02:30:43 dist-test-slave-dn4n krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303443, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:43.554595 30726 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:30:43.554988 30726 webserver.cc:460] Webserver started at http://127.11.220.194:34381/ using document root <none> and password file <none>
I20260925 02:30:43.555179 30726 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:43.555217 30726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:43.555375 30726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:43.556401 30726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "bd86dc2728434368a19386e18b826440"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "7a3b4ab33e4886477139145bd868631e"
server_key_iv: "fdaf035053bdcb456c58770b6b94ee63"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.556793 30726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "bd86dc2728434368a19386e18b826440"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "7a3b4ab33e4886477139145bd868631e"
server_key_iv: "fdaf035053bdcb456c58770b6b94ee63"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.558002 30726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:43.558702 30741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.558882 30726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:43.558966 30726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "bd86dc2728434368a19386e18b826440"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "7a3b4ab33e4886477139145bd868631e"
server_key_iv: "fdaf035053bdcb456c58770b6b94ee63"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.559017 30726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:43.564749 30726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:43.566285 30726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:43.566401 30726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:43.566565 30726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:43.566787 30726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:43.566812 30726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.566829 30726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:43.566840 30726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.571786 30726 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:35635
I20260925 02:30:43.571862 30854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:35635 every 8 connection(s)
I20260925 02:30:43.572037 30726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:30:43.576532 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 30726
I20260925 02:30:43.576654 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:30:43.576853 12147 external_mini_cluster.cc:1428] Setting key 501160991462ac6d5b133e71f2424934
Sep 25 02:30:43 dist-test-slave-dn4n krb5kdc[30481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303443, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:43.580658 30587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.572643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:57407 (local address 127.11.220.254:37207)
0925 02:30:43.572683 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:43.572685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:43.572857 (+ 172us) server_negotiation.cc:407] Connection header received
0925 02:30:43.573314 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:43.573316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:43.573329 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:43.573372 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:43.574152 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.574616 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.575503 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.575702 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.575770 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.577983 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:43.577984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:43.577984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:43.577989 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:43.579287 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:43.579752 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:43.579753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:43.579753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:43.579810 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:43.580032 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:43.580033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:43.580033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:43.580163 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:43.580226 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:43.580422 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 02:30:43.580523 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:30:43.580674 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.572619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:37207 (local address 127.11.220.194:57407)
0925 02:30:43.572753 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:43.572784 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:43.573215 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:43.573424 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:43.573427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:43.573893 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:43.574066 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.574070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.574704 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.574705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.575404 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.575406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.575777 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.575778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.575885 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.576310 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:43.576316 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:43.577900 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:43.579362 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:43.579369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:43.579371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:43.579679 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:43.579872 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:43.579873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:43.579873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:43.579967 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:43.580221 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:43.580225 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:43.580313 (+ 88us) client_negotiation.cc:769] Sending connection context
0925 02:30:43.580381 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 02:30:43.580545 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
I20260925 02:30:43.581180 30856 heartbeater.cc:347] Connected to a master server at 127.11.220.254:37207
I20260925 02:30:43.581333 30856 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:43.581502 30856 heartbeater.cc:511] Master 127.11.220.254:37207 requested a full tablet report, sending...
I20260925 02:30:43.581851 30526 ts_manager.cc:194] Registered new tserver with Master: bd86dc2728434368a19386e18b826440 (127.11.220.194:35635)
I20260925 02:30:43.582386 30526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:57407
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:30:43.599069 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37207
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:33085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:43.671689 30861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:43.671907 30861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:43.671941 30861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:43.673594 30861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:43.673647 30861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:43.673688 30861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:30:43.674945 30861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:33085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:37207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:43.675170 30861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:43.675356 30861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:43.679549 30867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:43.679562 30866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:43.679690 30861 server_base.cc:1034] running on GCE node
W20260925 02:30:43.679833 30869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:43.680055 30861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:43.680306 30861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:43.681465 30861 hybrid_clock.cc:648] HybridClock initialized: now 1790303443681449 us; error 43 us; skew 500 ppm
Sep 25 02:30:43 dist-test-slave-dn4n krb5kdc[30481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303443, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:43.683586 30861 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:30:43.683995 30861 webserver.cc:460] Webserver started at http://127.11.220.195:38337/ using document root <none> and password file <none>
I20260925 02:30:43.684199 30861 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:43.684258 30861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:43.684376 30861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:43.685261 30861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "f4587f4c3a614b9aaa99a51d74fe75cc"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "73929d964412e391666f2620921ac2ea"
server_key_iv: "fe50e2add557777836a582b7ccacc37c"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.685611 30861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "f4587f4c3a614b9aaa99a51d74fe75cc"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "73929d964412e391666f2620921ac2ea"
server_key_iv: "fe50e2add557777836a582b7ccacc37c"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.686865 30861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 02:30:43.687577 30876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.687721 30861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.687796 30861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "f4587f4c3a614b9aaa99a51d74fe75cc"
format_stamp: "Formatted at 2026-09-25 02:30:43 on dist-test-slave-dn4n"
server_key: "73929d964412e391666f2620921ac2ea"
server_key_iv: "fe50e2add557777836a582b7ccacc37c"
server_key_version: "encryptionkey@0"
I20260925 02:30:43.687842 30861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:43.704638 30861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:43.706115 30861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:43.706205 30861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:43.706349 30861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:43.706635 30861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:43.706666 30861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.706686 30861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:43.706732 30861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:43.712158 30861 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:36601
I20260925 02:30:43.712204 30994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:36601 every 8 connection(s)
I20260925 02:30:43.712510 30861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:30:43.712898 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 30861
I20260925 02:30:43.713016 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:30:43.713238 12147 external_mini_cluster.cc:1428] Setting key 59b8b7bc6e38c9bb4c450c0ab830e8c0
Sep 25 02:30:43 dist-test-slave-dn4n krb5kdc[30481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303443, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:43.720844 30587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.713114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:58875 (local address 127.11.220.254:37207)
0925 02:30:43.713169 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:43.713171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:43.713366 (+ 195us) server_negotiation.cc:407] Connection header received
0925 02:30:43.713982 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:43.713983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:43.713995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:43.714056 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:43.714894 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.715353 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.716255 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.716440 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.716530 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.718404 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:43.718405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:43.718405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:43.718412 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:43.719463 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:43.719828 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:43.719829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:43.719829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:43.719886 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:43.720164 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:43.720165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:43.720166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:43.720288 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:43.720361 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:43.720687 (+ 326us) server_negotiation.cc:299] Negotiation successful
0925 02:30:43.720760 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 02:30:43.721014 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.713090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:37207 (local address 127.11.220.195:58875)
0925 02:30:43.713241 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:43.713284 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:43.713867 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:43.714113 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:43.714116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:43.714607 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:43.714797 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.714801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.715448 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.715449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.716161 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.716163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.716538 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.716539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.716665 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.717053 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:43.717061 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:43.718318 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:43.719553 (+ 1235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:43.719560 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:43.719562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:43.719758 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:43.719957 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:43.719958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:43.719959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:43.720081 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:43.720353 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:43.720360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:43.720547 (+ 187us) client_negotiation.cc:769] Sending connection context
0925 02:30:43.720664 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 02:30:43.720806 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260925 02:30:43.721463 30995 heartbeater.cc:347] Connected to a master server at 127.11.220.254:37207
I20260925 02:30:43.721549 30995 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:43.721711 30995 heartbeater.cc:511] Master 127.11.220.254:37207 requested a full tablet report, sending...
I20260925 02:30:43.722043 30526 ts_manager.cc:194] Registered new tserver with Master: f4587f4c3a614b9aaa99a51d74fe75cc (127.11.220.195:36601)
I20260925 02:30:43.722371 30526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:58875
I20260925 02:30:43.724694 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:30:43.731006 30587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.725917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59802 (local address 127.11.220.254:37207)
0925 02:30:43.725950 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:43.725951 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:43.726053 (+ 102us) server_negotiation.cc:407] Connection header received
0925 02:30:43.726092 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:43.726092 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:43.726099 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:43.726143 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:43.726592 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.727001 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.727672 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.727838 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.727887 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.729010 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:43.729011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:43.729011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:43.729016 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:43.730214 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:43.730512 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:43.730512 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:43.730513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:43.730554 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:43.730690 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:43.730691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:43.730691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:43.730775 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:43.730834 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:43.730922 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 02:30:43.730960 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 02:30:43.732332 30526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:30:43.733870 30526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:30:43.739470 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.735782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:46501 (local address 127.0.0.1:50568)
0925 02:30:43.736359 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:43.736368 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:43.736420 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:43.736686 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:43.736687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:43.736689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:43.736721 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.736722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.737814 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.737815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.738616 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.738619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.739213 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.739214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.739286 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.739297 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 02:30:43.739361 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:30:43.739406 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":68,"threads_started":1}
I20260925 02:30:43.739483 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.735904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.11.220.193:46501)
0925 02:30:43.736100 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:43.736105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:43.736419 (+ 314us) server_negotiation.cc:407] Connection header received
0925 02:30:43.736512 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:43.736514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:43.736529 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:43.736607 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:43.736787 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.737731 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.738694 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.739131 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.739214 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.739265 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:43.739374 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 02:30:43.739412 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":75,"threads_started":1}
I20260925 02:30:43.739645 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.735692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55232 (local address 127.11.220.195:36601)
0925 02:30:43.735924 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:43.735930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:43.735966 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:30:43.736034 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:43.736036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:43.736048 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:43.736114 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:43.736263 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.737096 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.738532 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.739158 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.739224 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.739270 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:43.739521 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 02:30:43.739563 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
I20260925 02:30:43.739687 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.735620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:36601 (local address 127.0.0.1:55232)
0925 02:30:43.735876 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:43.735912 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:43.735978 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:43.736136 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:43.736138 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:43.736140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:43.736191 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.736192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.737194 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.737195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.738339 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.738341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.739303 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.739304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.739366 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.739369 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:43.739411 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 02:30:43.739458 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":75,"threads_started":1}
I20260925 02:30:43.739956 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.735717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.11.220.194:35635)
0925 02:30:43.736104 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:43.736107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:43.736116 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:30:43.736149 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:43.736151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:43.736164 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:43.736250 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:43.736366 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.737407 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.738605 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.739232 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.739270 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.739309 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:43.739489 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 02:30:43.739867 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":64,"threads_started":1}
I20260925 02:30:43.740185 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.735622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:35635 (local address 127.0.0.1:37512)
0925 02:30:43.735901 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:43.735917 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:43.735970 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:43.736231 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:43.736232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:43.736234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:43.736268 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.736269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.737514 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.737514 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.738522 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.738525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.739357 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.739358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.739412 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.739415 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:43.740093 (+ 678us) client_negotiation.cc:240] Negotiation successful
0925 02:30:43.740139 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":2,"thread_start_us":40,"threads_started":1}
I20260925 02:30:43.740130 30653 tablet_service.cc:1467] Processing CreateTablet for tablet 23a9bfaeddfa471aaaf38732ca89c391 (DEFAULT_TABLE table=test-table [id=39a2ed12b08148018a689a7167c8c4cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:43.740234 30926 tablet_service.cc:1467] Processing CreateTablet for tablet 23a9bfaeddfa471aaaf38732ca89c391 (DEFAULT_TABLE table=test-table [id=39a2ed12b08148018a689a7167c8c4cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:43.740464 30653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23a9bfaeddfa471aaaf38732ca89c391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:43.740502 30926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23a9bfaeddfa471aaaf38732ca89c391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:43.740787 30789 tablet_service.cc:1467] Processing CreateTablet for tablet 23a9bfaeddfa471aaaf38732ca89c391 (DEFAULT_TABLE table=test-table [id=39a2ed12b08148018a689a7167c8c4cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:43.741094 30789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23a9bfaeddfa471aaaf38732ca89c391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:43.743455 31011 tablet_bootstrap.cc:492] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6: Bootstrap starting.
I20260925 02:30:43.743546 31010 tablet_bootstrap.cc:492] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: Bootstrap starting.
I20260925 02:30:43.743644 31012 tablet_bootstrap.cc:492] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: Bootstrap starting.
I20260925 02:30:43.744174 31010 tablet_bootstrap.cc:654] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:43.744175 31012 tablet_bootstrap.cc:654] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:43.744793 31010 log.cc:824] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:43.744789 31012 log.cc:824] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:43.745046 31011 tablet_bootstrap.cc:654] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:43.745486 31012 tablet_bootstrap.cc:492] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: No bootstrap required, opened a new log
I20260925 02:30:43.745518 31010 tablet_bootstrap.cc:492] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: No bootstrap required, opened a new log
I20260925 02:30:43.745504 31011 log.cc:824] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:43.745556 31012 ts_tablet_manager.cc:1397] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 02:30:43.745565 31010 ts_tablet_manager.cc:1397] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:30:43.746130 31011 tablet_bootstrap.cc:492] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6: No bootstrap required, opened a new log
I20260925 02:30:43.746212 31011 ts_tablet_manager.cc:1397] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 02:30:43.747274 31010 raft_consensus.cc:348] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 } } peers { permanent_uuid: "bd86dc2728434368a19386e18b826440" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35635 } } peers { permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 } }
I20260925 02:30:43.747274 31012 raft_consensus.cc:348] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 } } peers { permanent_uuid: "bd86dc2728434368a19386e18b826440" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35635 } } peers { permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 } }
I20260925 02:30:43.747437 31012 raft_consensus.cc:374] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:43.747437 31010 raft_consensus.cc:374] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:43.747470 31012 raft_consensus.cc:729] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4587f4c3a614b9aaa99a51d74fe75cc, State: Initialized, Role: FOLLOWER
I20260925 02:30:43.747470 31010 raft_consensus.cc:729] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd86dc2728434368a19386e18b826440, State: Initialized, Role: FOLLOWER
I20260925 02:30:43.747589 31012 consensus_queue.cc:260] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 } } peers { permanent_uuid: "bd86dc2728434368a19386e18b826440" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35635 } } peers { permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 } }
I20260925 02:30:43.747589 31010 consensus_queue.cc:260] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 } } peers { permanent_uuid: "bd86dc2728434368a19386e18b826440" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35635 } } peers { permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 } }
I20260925 02:30:43.747684 31011 raft_consensus.cc:348] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 } } peers { permanent_uuid: "bd86dc2728434368a19386e18b826440" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35635 } } peers { permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 } }
I20260925 02:30:43.747846 31010 ts_tablet_manager.cc:1428] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:30:43.747846 31012 ts_tablet_manager.cc:1428] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 02:30:43.747848 31011 raft_consensus.cc:374] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:43.747888 31011 raft_consensus.cc:729] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4800a1c6175e4e04a6c6e1b3d8d03af6, State: Initialized, Role: FOLLOWER
I20260925 02:30:43.747995 31011 consensus_queue.cc:260] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 } } peers { permanent_uuid: "bd86dc2728434368a19386e18b826440" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35635 } } peers { permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 } }
I20260925 02:30:43.748100 30995 heartbeater.cc:503] Master 127.11.220.254:37207 was elected leader, sending a full tablet report...
I20260925 02:30:43.748251 31011 ts_tablet_manager.cc:1428] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:30:43.748307 30721 heartbeater.cc:503] Master 127.11.220.254:37207 was elected leader, sending a full tablet report...
I20260925 02:30:43.748338 30856 heartbeater.cc:503] Master 127.11.220.254:37207 was elected leader, sending a full tablet report...
W20260925 02:30:43.822477 30857 tablet.cc:2369] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:30:43.945593 30722 tablet.cc:2369] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:30:43.958997 31017 raft_consensus.cc:482] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:43.959147 31017 raft_consensus.cc:504] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 } } peers { permanent_uuid: "bd86dc2728434368a19386e18b826440" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35635 } } peers { permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 } }
I20260925 02:30:43.959528 31017 leader_election.cc:288] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4587f4c3a614b9aaa99a51d74fe75cc (127.11.220.195:36601), 4800a1c6175e4e04a6c6e1b3d8d03af6 (127.11.220.193:46501)
I20260925 02:30:43.962589 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.959915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:43705 (local address 127.11.220.193:46501)
0925 02:30:43.959975 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:43.959977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:43.960124 (+ 147us) server_negotiation.cc:407] Connection header received
0925 02:30:43.960151 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:43.960151 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:43.960164 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:43.960202 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:43.960364 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.960975 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.961881 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.962212 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.962289 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.962299 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:43.962458 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 02:30:43.962495 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 02:30:43.962570 31019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.959854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:46501 (local address 127.11.220.194:43705)
0925 02:30:43.960029 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:43.960035 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:43.960104 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:43.960226 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:43.960227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:43.960228 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:43.960282 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.960283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.961051 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.961052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.961787 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.961790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.962289 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.962291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.962366 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.962370 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:30:43.962424 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 02:30:43.962479 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260925 02:30:43.962623 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.959833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:52747 (local address 127.11.220.195:36601)
0925 02:30:43.959878 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:43.959881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:43.959891 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:30:43.959992 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:43.959994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:43.960015 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:43.960066 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:43.960263 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.960916 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.961819 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.962261 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.962342 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.962350 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:43.962514 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 02:30:43.962553 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 02:30:43.962673 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:43.959743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:36601 (local address 127.11.220.194:52747)
0925 02:30:43.959810 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:43.959824 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:43.959888 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:43.960089 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:43.960090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:43.960092 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:43.960165 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.960167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.960989 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.960991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.961733 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:43.961735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:43.962342 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:43.962343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:43.962419 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:43.962422 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:43.962514 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 02:30:43.962554 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 02:30:43.963080 30996 tablet.cc:2369] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:30:43.963045 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a9bfaeddfa471aaaf38732ca89c391" candidate_uuid: "bd86dc2728434368a19386e18b826440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" is_pre_election: true
I20260925 02:30:43.963078 30946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a9bfaeddfa471aaaf38732ca89c391" candidate_uuid: "bd86dc2728434368a19386e18b826440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" is_pre_election: true
I20260925 02:30:43.963202 30674 raft_consensus.cc:2454] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd86dc2728434368a19386e18b826440 in term 0.
I20260925 02:30:43.963270 30946 raft_consensus.cc:2454] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd86dc2728434368a19386e18b826440 in term 0.
I20260925 02:30:43.963407 30745 leader_election.cc:302] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4800a1c6175e4e04a6c6e1b3d8d03af6, bd86dc2728434368a19386e18b826440; no voters:
I20260925 02:30:43.963548 31017 raft_consensus.cc:2781] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:30:43.963621 31017 raft_consensus.cc:482] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:30:43.963639 31017 raft_consensus.cc:3037] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:43.964323 31017 raft_consensus.cc:504] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 } } peers { permanent_uuid: "bd86dc2728434368a19386e18b826440" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35635 } } peers { permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 } }
I20260925 02:30:43.964461 31017 leader_election.cc:288] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [CANDIDATE]: Term 1 election: Requested vote from peers f4587f4c3a614b9aaa99a51d74fe75cc (127.11.220.195:36601), 4800a1c6175e4e04a6c6e1b3d8d03af6 (127.11.220.193:46501)
I20260925 02:30:43.964594 30946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a9bfaeddfa471aaaf38732ca89c391" candidate_uuid: "bd86dc2728434368a19386e18b826440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc"
I20260925 02:30:43.964607 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23a9bfaeddfa471aaaf38732ca89c391" candidate_uuid: "bd86dc2728434368a19386e18b826440" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6"
I20260925 02:30:43.964670 30946 raft_consensus.cc:3037] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:43.964671 30674 raft_consensus.cc:3037] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:43.965485 30674 raft_consensus.cc:2454] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd86dc2728434368a19386e18b826440 in term 1.
I20260925 02:30:43.965485 30946 raft_consensus.cc:2454] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd86dc2728434368a19386e18b826440 in term 1.
I20260925 02:30:43.965641 30745 leader_election.cc:302] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4800a1c6175e4e04a6c6e1b3d8d03af6, bd86dc2728434368a19386e18b826440; no voters:
I20260925 02:30:43.965724 31017 raft_consensus.cc:2781] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:43.965852 31017 raft_consensus.cc:686] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 1 LEADER]: Becoming Leader. State: Replica: bd86dc2728434368a19386e18b826440, State: Running, Role: LEADER
I20260925 02:30:43.965972 31017 consensus_queue.cc:237] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 } } peers { permanent_uuid: "bd86dc2728434368a19386e18b826440" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35635 } } peers { permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 } }
I20260925 02:30:43.966862 30525 catalog_manager.cc:5495] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd86dc2728434368a19386e18b826440 (127.11.220.194). New cstate: current_term: 1 leader_uuid: "bd86dc2728434368a19386e18b826440" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd86dc2728434368a19386e18b826440" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 35635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 } health_report { overall_health: UNKNOWN } } }
I20260925 02:30:44.020641 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.018728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.11.220.194:35635)
0925 02:30:44.018769 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.018771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.018780 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:30:44.018817 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.018818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.018835 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.018894 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:30:44.018989 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.019408 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.020050 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.020214 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.020260 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.020360 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:30:44.020437 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:30:44.020526 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.020559 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.020567 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:30:44.022464 30674 raft_consensus.cc:1261] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6 [term 1 FOLLOWER]: Refusing update from remote peer bd86dc2728434368a19386e18b826440: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:44.022469 30946 raft_consensus.cc:1261] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc [term 1 FOLLOWER]: Refusing update from remote peer bd86dc2728434368a19386e18b826440: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:44.022836 31017 consensus_queue.cc:1035] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4800a1c6175e4e04a6c6e1b3d8d03af6" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 46501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:44.022980 31020 consensus_queue.cc:1035] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4587f4c3a614b9aaa99a51d74fe75cc" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 36601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:44.025424 31025 mvcc.cc:205] Tried to move back new op lower bound from 7333082906713255936 to 7333082906485288960. Current Snapshot: MvccSnapshot[applied={T|T < 7333082906713255936}]
I20260925 02:30:44.027733 30525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:44.027832 30525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:44.028584 30525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 229ba3f94ff04524b0fd2d2a4313a3dd: Sending DeleteTablet for 3 replicas of tablet 23a9bfaeddfa471aaaf38732ca89c391
I20260925 02:30:44.028923 30926 tablet_service.cc:1514] Processing DeleteTablet for tablet 23a9bfaeddfa471aaaf38732ca89c391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:44 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:55232
I20260925 02:30:44.028965 30653 tablet_service.cc:1514] Processing DeleteTablet for tablet 23a9bfaeddfa471aaaf38732ca89c391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:44 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:50568
I20260925 02:30:44.029117 30789 tablet_service.cc:1514] Processing DeleteTablet for tablet 23a9bfaeddfa471aaaf38732ca89c391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:44 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:37512
I20260925 02:30:44.029253 31035 tablet_replica.cc:330] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6: stopping tablet replica
I20260925 02:30:44.029279 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 30590
I20260925 02:30:44.029366 31034 tablet_replica.cc:330] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: stopping tablet replica
I20260925 02:30:44.029413 31035 raft_consensus.cc:2229] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:44.029456 31034 raft_consensus.cc:2229] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:44.029523 31035 raft_consensus.cc:2258] T 23a9bfaeddfa471aaaf38732ca89c391 P 4800a1c6175e4e04a6c6e1b3d8d03af6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:44.029520 31036 tablet_replica.cc:330] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: stopping tablet replica
I20260925 02:30:44.029538 31034 raft_consensus.cc:2258] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:44.029667 31036 raft_consensus.cc:2229] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:30:44.029820 31036 raft_consensus.cc:2258] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:44.030300 31036 ts_tablet_manager.cc:1905] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:44.032311 31036 ts_tablet_manager.cc:1918] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:44.032390 31036 log.cc:1196] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/23a9bfaeddfa471aaaf38732ca89c391
I20260925 02:30:44.032686 31036 ts_tablet_manager.cc:1939] T 23a9bfaeddfa471aaaf38732ca89c391 P bd86dc2728434368a19386e18b826440: Deleting consensus metadata
I20260925 02:30:44.033110 30510 catalog_manager.cc:4841] TS bd86dc2728434368a19386e18b826440 (127.11.220.194:35635): tablet 23a9bfaeddfa471aaaf38732ca89c391 (table test-table [id=39a2ed12b08148018a689a7167c8c4cd]) successfully deleted
I20260925 02:30:44.033371 31034 ts_tablet_manager.cc:1905] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:44.034740 31034 ts_tablet_manager.cc:1918] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:44.034814 31034 log.cc:1196] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/23a9bfaeddfa471aaaf38732ca89c391
I20260925 02:30:44.035039 31034 ts_tablet_manager.cc:1939] T 23a9bfaeddfa471aaaf38732ca89c391 P f4587f4c3a614b9aaa99a51d74fe75cc: Deleting consensus metadata
I20260925 02:30:44.035405 30513 catalog_manager.cc:4841] TS f4587f4c3a614b9aaa99a51d74fe75cc (127.11.220.195:36601): tablet 23a9bfaeddfa471aaaf38732ca89c391 (table test-table [id=39a2ed12b08148018a689a7167c8c4cd]) successfully deleted
W20260925 02:30:44.037650 30513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:46501 (error 108)
I20260925 02:30:44.038128 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.037999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:46501 (local address 127.0.0.1:50576)
0925 02:30:44.038056 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:44.038092 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:46501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 02:30:44.038235 30513 catalog_manager.cc:4584] TS 4800a1c6175e4e04a6c6e1b3d8d03af6 (127.11.220.193:46501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23a9bfaeddfa471aaaf38732ca89c391: Network error: Client connection negotiation failed: client connection to 127.11.220.193:46501: connect: Connection refused (error 111)
I20260925 02:30:44.038376 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 30726
I20260925 02:30:44.043447 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 30861
I20260925 02:30:44.047619 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 30494
2026-09-25T02:30:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1937 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T02:30:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:44Z Disabled control of system clock
I20260925 02:30:44.073644 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:35493
--webserver_interface=127.11.220.254
--webserver_port=0
--builtin_ntp_servers=127.11.220.212:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:35493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:30:44.147187 31041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:44.147359 31041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:44.147377 31041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:44.148335 31041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:44.148365 31041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:44.148375 31041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:44.148386 31041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:44.148397 31041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:44.149552 31041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:35493
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:35493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:44.149724 31041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:44.149919 31041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:44.154026 31047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:44.154017 31046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:44.154104 31041 server_base.cc:1034] running on GCE node
W20260925 02:30:44.154003 31049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:44.154474 31041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:44.154745 31041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:44.155887 31041 hybrid_clock.cc:648] HybridClock initialized: now 1790303444155875 us; error 37 us; skew 500 ppm
I20260925 02:30:44.156971 31041 webserver.cc:460] Webserver started at http://127.11.220.254:34157/ using document root <none> and password file <none>
I20260925 02:30:44.157191 31041 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:44.157251 31041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:44.157366 31041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:44.158268 31041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "3fb1a81d337440babc208a5f61017b92"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "2e6340ecb06a7bbb3214c2876b39f7e4"
server_key_iv: "1faebe84ebe333b0825afd351123fa71"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.158581 31041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "3fb1a81d337440babc208a5f61017b92"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "2e6340ecb06a7bbb3214c2876b39f7e4"
server_key_iv: "1faebe84ebe333b0825afd351123fa71"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.159628 31041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:30:44.160264 31055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:44.160418 31041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:44.160521 31041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "3fb1a81d337440babc208a5f61017b92"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "2e6340ecb06a7bbb3214c2876b39f7e4"
server_key_iv: "1faebe84ebe333b0825afd351123fa71"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.160605 31041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:44.171525 31041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:44.171767 31041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:44.171921 31041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:44.176216 31041 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:35493
I20260925 02:30:44.176244 31110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:35493 every 8 connection(s)
I20260925 02:30:44.176602 31041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:44.177277 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 31041
I20260925 02:30:44.177412 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:44.177565 31111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:44.177636 12147 external_mini_cluster.cc:1428] Setting key 04496ac69a405191183ee8ad4113ddce
I20260925 02:30:44.180292 31111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92: Bootstrap starting.
I20260925 02:30:44.180912 31111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:44.181216 31111 log.cc:824] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:44.181952 31111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92: No bootstrap required, opened a new log
I20260925 02:30:44.183103 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.178158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56976 (local address 127.11.220.254:35493)
0925 02:30:44.178347 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.178353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.178375 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:30:44.178858 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.178863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.178917 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.179020 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:44.179245 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.180043 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.181098 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.181640 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.181741 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.182418 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:44.182422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:44.182423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:44.182585 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:44.182611 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:44.182620 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:44.182667 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.182858 (+ 191us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.182936 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260925 02:30:44.183924 31111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb1a81d337440babc208a5f61017b92" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35493 } }
I20260925 02:30:44.184049 31111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:44.184070 31111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb1a81d337440babc208a5f61017b92, State: Initialized, Role: FOLLOWER
I20260925 02:30:44.184192 31111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb1a81d337440babc208a5f61017b92" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35493 } }
I20260925 02:30:44.184255 31111 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:44.184294 31111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:44.184336 31111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:44.184834 31111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb1a81d337440babc208a5f61017b92" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35493 } }
I20260925 02:30:44.184950 31111 leader_election.cc:302] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fb1a81d337440babc208a5f61017b92; no voters:
I20260925 02:30:44.185096 31111 leader_election.cc:288] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:44.185158 31116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:44.185309 31116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb1a81d337440babc208a5f61017b92, State: Running, Role: LEADER
I20260925 02:30:44.185351 31111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:44.185388 31116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb1a81d337440babc208a5f61017b92" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35493 } }
I20260925 02:30:44.185750 31117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fb1a81d337440babc208a5f61017b92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb1a81d337440babc208a5f61017b92" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35493 } } }
I20260925 02:30:44.185760 31118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fb1a81d337440babc208a5f61017b92. Latest consensus state: current_term: 1 leader_uuid: "3fb1a81d337440babc208a5f61017b92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fb1a81d337440babc208a5f61017b92" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35493 } } }
I20260925 02:30:44.185878 31117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:44.186368 31118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:44.186988 31129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:44.187471 31129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:44.188730 31129 catalog_manager.cc:1347] Generated new cluster ID: f5a0a477378c4fb0bfb399904c27d968
I20260925 02:30:44.188781 31129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:44.200830 31129 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:44.201259 31129 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:44.208308 31129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92: Generated new TSK 0
I20260925 02:30:44.208501 31129 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:44.212605 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35493
--builtin_ntp_servers=127.11.220.212:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:30:44.286211 31135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:44.286374 31135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:44.286405 31135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:44.287550 31135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:44.287590 31135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:44.287623 31135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:30:44.288900 31135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:44.289117 31135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:44.289390 31135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:44.294106 31141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:44.294127 31142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:44.294288 31135 server_base.cc:1034] running on GCE node
W20260925 02:30:44.294128 31144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:44.294559 31135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:44.294772 31135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:44.295921 31135 hybrid_clock.cc:648] HybridClock initialized: now 1790303444295912 us; error 28 us; skew 500 ppm
I20260925 02:30:44.297020 31135 webserver.cc:460] Webserver started at http://127.11.220.193:35393/ using document root <none> and password file <none>
I20260925 02:30:44.297236 31135 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:44.297292 31135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:44.297405 31135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:44.298254 31135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "c33e9822ee23478bb9d862962c625412"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "91022150fb3c9114de96a298d9ebd339"
server_key_iv: "f5ee8294c234f1de41db8aba3cd668b0"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.298557 31135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "c33e9822ee23478bb9d862962c625412"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "91022150fb3c9114de96a298d9ebd339"
server_key_iv: "f5ee8294c234f1de41db8aba3cd668b0"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.299696 31135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:30:44.300370 31150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:44.300572 31135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:44.300657 31135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "c33e9822ee23478bb9d862962c625412"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "91022150fb3c9114de96a298d9ebd339"
server_key_iv: "f5ee8294c234f1de41db8aba3cd668b0"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.300731 31135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:44.316642 31135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:44.316890 31135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:44.317026 31135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:44.317250 31135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:44.317534 31135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:44.317582 31135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:44.317619 31135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:44.317644 31135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:44.322867 31135 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:40345
I20260925 02:30:44.322963 31264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:40345 every 8 connection(s)
I20260925 02:30:44.323220 31135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:30:44.327101 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 31135
I20260925 02:30:44.327227 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:30:44.327426 12147 external_mini_cluster.cc:1428] Setting key bb280b7ad116bb3ef4bc88b2f3c1f913
I20260925 02:30:44.327906 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.323855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:33489 (local address 127.11.220.254:35493)
0925 02:30:44.323917 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.323919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.324115 (+ 196us) server_negotiation.cc:407] Connection header received
0925 02:30:44.324576 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.324577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.324591 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.324657 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:44.325264 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.325721 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.326616 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.326784 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.326830 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.327645 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:44.327646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:44.327647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:44.327650 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:44.327661 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:44.327664 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:44.327723 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.327805 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.327837 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:30:44.328088 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.323846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35493 (local address 127.11.220.193:33489)
0925 02:30:44.324012 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:44.324039 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:44.324482 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:44.324718 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:44.324722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:44.324731 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:44.325148 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.325152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.325846 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.325847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.326548 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.326550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.326848 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.326849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.326943 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.327472 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:30:44.327472 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:44.327512 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:30:44.327541 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:44.327736 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:44.327743 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:44.327746 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:44.327799 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 02:30:44.327890 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260925 02:30:44.328271 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35493
--builtin_ntp_servers=127.11.220.212:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 02:30:44.328498 31265 heartbeater.cc:347] Connected to a master server at 127.11.220.254:35493
I20260925 02:30:44.328575 31265 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:44.328725 31265 heartbeater.cc:511] Master 127.11.220.254:35493 requested a full tablet report, sending...
I20260925 02:30:44.329206 31073 ts_manager.cc:194] Registered new tserver with Master: c33e9822ee23478bb9d862962c625412 (127.11.220.193:40345)
I20260925 02:30:44.329751 31073 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.193:33489
W20260925 02:30:44.400838 31269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:44.401018 31269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:44.401037 31269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:44.402205 31269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:44.402238 31269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:44.402261 31269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:30:44.403569 31269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:44.403802 31269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:44.404098 31269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:30:44.408757 31269 server_base.cc:1034] running on GCE node
W20260925 02:30:44.408690 31277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:44.408694 31274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:44.408710 31275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:44.409086 31269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:44.409307 31269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:44.410449 31269 hybrid_clock.cc:648] HybridClock initialized: now 1790303444410437 us; error 23 us; skew 500 ppm
I20260925 02:30:44.411434 31269 webserver.cc:460] Webserver started at http://127.11.220.194:34517/ using document root <none> and password file <none>
I20260925 02:30:44.411661 31269 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:44.411723 31269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:44.411870 31269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:44.412923 31269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "9770676d6b4d404382235e92dca32f59"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "172f3aaa18235abfb317a30434ccc98c"
server_key_iv: "37daf523d3a6a7fdbebbcdb98a795408"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.413269 31269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "9770676d6b4d404382235e92dca32f59"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "172f3aaa18235abfb317a30434ccc98c"
server_key_iv: "37daf523d3a6a7fdbebbcdb98a795408"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.414439 31269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:30:44.415117 31284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:44.415275 31269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:44.415370 31269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "9770676d6b4d404382235e92dca32f59"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "172f3aaa18235abfb317a30434ccc98c"
server_key_iv: "37daf523d3a6a7fdbebbcdb98a795408"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.415453 31269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:44.426146 31269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:44.426398 31269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:44.426537 31269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:44.426741 31269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:44.427029 31269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:44.427081 31269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:44.427134 31269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:44.427160 31269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:44.432559 31269 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:43459
I20260925 02:30:44.432623 31398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:43459 every 8 connection(s)
I20260925 02:30:44.432914 31269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:30:44.437400 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.433464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:39787 (local address 127.11.220.254:35493)
0925 02:30:44.433538 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.433540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.433711 (+ 171us) server_negotiation.cc:407] Connection header received
0925 02:30:44.434140 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.434141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.434152 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.434211 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:44.434784 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.435230 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.436130 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.436316 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.436401 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.437124 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:44.437125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:44.437125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:44.437130 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:44.437143 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:44.437147 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:44.437207 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.437293 (+ 86us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.437325 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 02:30:44.437647 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.433425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35493 (local address 127.11.220.194:39787)
0925 02:30:44.433602 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:44.433647 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:44.434041 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:44.434266 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:44.434269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:44.434277 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:44.434695 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.434698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.435316 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.435317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.436032 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.436034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.436413 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.436414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.436564 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.436966 (+ 402us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:30:44.436967 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:44.436990 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:30:44.437009 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:44.437207 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:44.437214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:44.437216 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:30:44.437295 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 02:30:44.437382 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260925 02:30:44.438108 31399 heartbeater.cc:347] Connected to a master server at 127.11.220.254:35493
I20260925 02:30:44.438175 31399 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:44.438354 31399 heartbeater.cc:511] Master 127.11.220.254:35493 requested a full tablet report, sending...
I20260925 02:30:44.438735 31073 ts_manager.cc:194] Registered new tserver with Master: 9770676d6b4d404382235e92dca32f59 (127.11.220.194:43459)
I20260925 02:30:44.439085 31073 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.194:39787
I20260925 02:30:44.442592 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 31269
I20260925 02:30:44.442721 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:30:44.442905 12147 external_mini_cluster.cc:1428] Setting key 3d05108032097095993d892e1ee6e3a6
I20260925 02:30:44.443778 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35493
--builtin_ntp_servers=127.11.220.212:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:30:44.517457 31402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:44.517736 31402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:44.517769 31402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:44.518819 31402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:44.518873 31402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:44.518911 31402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:30:44.520180 31402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:35493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:44.520416 31402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:44.520695 31402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:44.525043 31411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:44.525142 31402 server_base.cc:1034] running on GCE node
W20260925 02:30:44.525076 31409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:44.525069 31408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:44.525411 31402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:44.525674 31402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:44.526808 31402 hybrid_clock.cc:648] HybridClock initialized: now 1790303444526794 us; error 29 us; skew 500 ppm
I20260925 02:30:44.527894 31402 webserver.cc:460] Webserver started at http://127.11.220.195:44035/ using document root <none> and password file <none>
I20260925 02:30:44.528079 31402 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:44.528116 31402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:44.528193 31402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:44.529100 31402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "c4631b7592df47b99deedb78c0c345d6"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "67e1908008fbbb88f8f70a20999145bc"
server_key_iv: "fb9a8cbef2ca242399c5a0a738bfde74"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.529392 31402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "c4631b7592df47b99deedb78c0c345d6"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "67e1908008fbbb88f8f70a20999145bc"
server_key_iv: "fb9a8cbef2ca242399c5a0a738bfde74"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.530431 31402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:30:44.530993 31417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:44.531163 31402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:44.531239 31402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "c4631b7592df47b99deedb78c0c345d6"
format_stamp: "Formatted at 2026-09-25 02:30:44 on dist-test-slave-dn4n"
server_key: "67e1908008fbbb88f8f70a20999145bc"
server_key_iv: "fb9a8cbef2ca242399c5a0a738bfde74"
server_key_version: "encryptionkey@0"
I20260925 02:30:44.531286 31402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:44.547763 31402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:44.547974 31402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:44.548115 31402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:44.548292 31402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:44.548614 31402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:44.548645 31402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:44.548663 31402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:44.548702 31402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:44.553534 31402 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:34033
I20260925 02:30:44.553589 31532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:34033 every 8 connection(s)
I20260925 02:30:44.553817 31402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:30:44.558148 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 31402
I20260925 02:30:44.558256 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:30:44.558296 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.554366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:42009 (local address 127.11.220.254:35493)
0925 02:30:44.554419 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.554421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.554604 (+ 183us) server_negotiation.cc:407] Connection header received
0925 02:30:44.555011 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.555012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.555024 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.555067 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:44.555632 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.556125 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.557105 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.557307 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.557387 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.558044 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:44.558045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:44.558046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:44.558050 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:44.558065 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:44.558069 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:44.558116 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.558199 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.558224 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 02:30:44.558470 12147 external_mini_cluster.cc:1428] Setting key 4dcbbaaa22d191a2d2dd200ab3bb6f96
I20260925 02:30:44.558492 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.554355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35493 (local address 127.11.220.195:42009)
0925 02:30:44.554515 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:44.554540 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:44.554912 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:44.555136 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:44.555139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:44.555148 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:44.555547 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.555551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.556216 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.556217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.557007 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.557009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.557394 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.557395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.557526 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.557921 (+ 395us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:30:44.557921 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:44.557943 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:30:44.557961 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:44.558120 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:44.558125 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:44.558128 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:44.558201 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 02:30:44.558287 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
I20260925 02:30:44.558930 31533 heartbeater.cc:347] Connected to a master server at 127.11.220.254:35493
I20260925 02:30:44.559024 31533 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:44.559188 31533 heartbeater.cc:511] Master 127.11.220.254:35493 requested a full tablet report, sending...
I20260925 02:30:44.559463 31072 ts_manager.cc:194] Registered new tserver with Master: c4631b7592df47b99deedb78c0c345d6 (127.11.220.195:34033)
I20260925 02:30:44.559753 31072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.195:42009
I20260925 02:30:44.569941 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:30:44.576996 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.571143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56982 (local address 127.11.220.254:35493)
0925 02:30:44.571210 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.571212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.571259 (+ 47us) server_negotiation.cc:407] Connection header received
0925 02:30:44.571291 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.571291 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.571298 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.571335 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:44.571440 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.571857 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.572609 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.572773 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.572821 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.576703 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:44.576704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:44.576704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:44.576708 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:44.576718 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:44.576721 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:44.576784 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.576892 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.576904 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 02:30:44.578298 31072 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:30:44.579684 31072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:30:44.585335 31543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.581467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:34033 (local address 127.0.0.1:48582)
0925 02:30:44.581628 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:44.581658 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:44.581714 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:44.582112 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:44.582113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:44.582116 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:44.582172 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.582173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.583310 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.583311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.584131 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.584133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.585056 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.585057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.585125 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.585136 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 02:30:44.585214 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 02:30:44.585258 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":4,"thread_start_us":53,"threads_started":1}
I20260925 02:30:44.585532 31542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.581467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:43459 (local address 127.0.0.1:32978)
0925 02:30:44.581816 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:44.581824 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:44.581864 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:44.582060 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:44.582062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:44.582064 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:44.582114 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.582115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.583310 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.583310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.584493 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.584495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.585294 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.585295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.585350 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.585353 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:44.585404 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 02:30:44.585438 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":65,"spinlock_wait_cycles":384,"thread_start_us":86,"threads_started":1}
I20260925 02:30:44.585647 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.581627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48582 (local address 127.11.220.195:34033)
0925 02:30:44.581912 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.581915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.581923 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:30:44.581954 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.581956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.581968 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.582054 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:44.582232 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.583221 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.584258 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.584974 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.585044 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.585114 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.585534 (+ 420us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.585585 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":59,"threads_started":1}
I20260925 02:30:44.585785 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.581634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32978 (local address 127.11.220.194:43459)
0925 02:30:44.581819 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.581824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.581915 (+ 91us) server_negotiation.cc:407] Connection header received
0925 02:30:44.581955 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.581957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.581972 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.582043 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:44.582188 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.583225 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.584562 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.585234 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.585294 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.585364 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.585500 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.585550 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":82,"threads_started":1}
I20260925 02:30:44.586179 31466 tablet_service.cc:1467] Processing CreateTablet for tablet a7692e8054df40e48800faa8a49dd745 (DEFAULT_TABLE table=test-table [id=efa40a34f8d642d793386aac81be9162]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:44.586337 31332 tablet_service.cc:1467] Processing CreateTablet for tablet a7692e8054df40e48800faa8a49dd745 (DEFAULT_TABLE table=test-table [id=efa40a34f8d642d793386aac81be9162]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:44.586485 31466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7692e8054df40e48800faa8a49dd745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:44.586558 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.581482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:40345 (local address 127.0.0.1:46676)
0925 02:30:44.581862 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:44.581870 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:44.581920 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:44.582123 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:44.582124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:44.582125 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:44.582173 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.582173 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.584455 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.584456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.585249 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.585251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.586337 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.586338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.586393 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.586396 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:44.586443 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 02:30:44.586477 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":4,"thread_start_us":82,"threads_started":1}
I20260925 02:30:44.586688 31332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7692e8054df40e48800faa8a49dd745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:44.586752 31546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.581706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.11.220.193:40345)
0925 02:30:44.581878 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.581883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.581914 (+ 31us) server_negotiation.cc:407] Connection header received
0925 02:30:44.581980 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.581982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.581996 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.582058 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:44.583312 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.584364 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.585322 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.586133 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.586200 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.586272 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.586644 (+ 372us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.586683 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260925 02:30:44.587345 31198 tablet_service.cc:1467] Processing CreateTablet for tablet a7692e8054df40e48800faa8a49dd745 (DEFAULT_TABLE table=test-table [id=efa40a34f8d642d793386aac81be9162]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:44.587677 31198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7692e8054df40e48800faa8a49dd745. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:44.589557 31548 tablet_bootstrap.cc:492] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: Bootstrap starting.
I20260925 02:30:44.589752 31547 tablet_bootstrap.cc:492] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: Bootstrap starting.
I20260925 02:30:44.590222 31548 tablet_bootstrap.cc:654] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:44.590476 31547 tablet_bootstrap.cc:654] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:44.590512 31548 log.cc:824] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:44.590735 31549 tablet_bootstrap.cc:492] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412: Bootstrap starting.
I20260925 02:30:44.590884 31547 log.cc:824] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:44.591267 31548 tablet_bootstrap.cc:492] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: No bootstrap required, opened a new log
I20260925 02:30:44.591336 31548 ts_tablet_manager.cc:1397] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:44.591408 31549 tablet_bootstrap.cc:654] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:44.591569 31547 tablet_bootstrap.cc:492] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: No bootstrap required, opened a new log
I20260925 02:30:44.591641 31547 ts_tablet_manager.cc:1397] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:44.591710 31549 log.cc:824] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:44.592478 31549 tablet_bootstrap.cc:492] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412: No bootstrap required, opened a new log
I20260925 02:30:44.592542 31549 ts_tablet_manager.cc:1397] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:30:44.593798 31547 raft_consensus.cc:348] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 } } peers { permanent_uuid: "9770676d6b4d404382235e92dca32f59" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 43459 } } peers { permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 } }
I20260925 02:30:44.593930 31547 raft_consensus.cc:374] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:44.593971 31547 raft_consensus.cc:729] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9770676d6b4d404382235e92dca32f59, State: Initialized, Role: FOLLOWER
I20260925 02:30:44.593923 31548 raft_consensus.cc:348] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 } } peers { permanent_uuid: "9770676d6b4d404382235e92dca32f59" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 43459 } } peers { permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 } }
I20260925 02:30:44.594058 31548 raft_consensus.cc:374] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:44.594091 31548 raft_consensus.cc:729] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4631b7592df47b99deedb78c0c345d6, State: Initialized, Role: FOLLOWER
I20260925 02:30:44.594105 31547 consensus_queue.cc:260] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 } } peers { permanent_uuid: "9770676d6b4d404382235e92dca32f59" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 43459 } } peers { permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 } }
I20260925 02:30:44.594185 31548 consensus_queue.cc:260] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 } } peers { permanent_uuid: "9770676d6b4d404382235e92dca32f59" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 43459 } } peers { permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 } }
I20260925 02:30:44.594375 31547 ts_tablet_manager.cc:1428] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:30:44.594401 31548 ts_tablet_manager.cc:1428] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:30:44.594529 31533 heartbeater.cc:503] Master 127.11.220.254:35493 was elected leader, sending a full tablet report...
I20260925 02:30:44.594763 31399 heartbeater.cc:503] Master 127.11.220.254:35493 was elected leader, sending a full tablet report...
I20260925 02:30:44.594910 31549 raft_consensus.cc:348] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 } } peers { permanent_uuid: "9770676d6b4d404382235e92dca32f59" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 43459 } } peers { permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 } }
I20260925 02:30:44.595047 31549 raft_consensus.cc:374] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:44.595089 31549 raft_consensus.cc:729] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c33e9822ee23478bb9d862962c625412, State: Initialized, Role: FOLLOWER
I20260925 02:30:44.595185 31549 consensus_queue.cc:260] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 } } peers { permanent_uuid: "9770676d6b4d404382235e92dca32f59" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 43459 } } peers { permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 } }
I20260925 02:30:44.595407 31549 ts_tablet_manager.cc:1428] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:30:44.595449 31265 heartbeater.cc:503] Master 127.11.220.254:35493 was elected leader, sending a full tablet report...
W20260925 02:30:44.683393 31400 tablet.cc:2369] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:30:44.804294 31534 tablet.cc:2369] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:30:44.804896 31553 raft_consensus.cc:482] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:44.805024 31553 raft_consensus.cc:504] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 } } peers { permanent_uuid: "9770676d6b4d404382235e92dca32f59" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 43459 } } peers { permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 } }
I20260925 02:30:44.805390 31553 leader_election.cc:288] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4631b7592df47b99deedb78c0c345d6 (127.11.220.195:34033), c33e9822ee23478bb9d862962c625412 (127.11.220.193:40345)
I20260925 02:30:44.808132 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.805543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:40345 (local address 127.11.220.194:50181)
0925 02:30:44.805577 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:44.805588 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:44.805645 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:44.805813 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:44.805814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:44.805817 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:44.805889 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.805891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.806578 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.806579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.807333 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.807335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.807866 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.807867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.807931 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.807935 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:30:44.807991 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 02:30:44.808050 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260925 02:30:44.808153 31546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.805610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:50181 (local address 127.11.220.193:40345)
0925 02:30:44.805676 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.805678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.805688 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:30:44.805716 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.805716 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.805729 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.805780 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:44.805949 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.806499 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.807408 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.807792 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.807841 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.807846 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.808000 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.808059 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 02:30:44.808370 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.805634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:53583 (local address 127.11.220.195:34033)
0925 02:30:44.805686 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.805688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.805926 (+ 238us) server_negotiation.cc:407] Connection header received
0925 02:30:44.806015 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.806016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.806025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.806072 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:44.806208 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.806710 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.807594 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.807974 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.808036 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.808043 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.808266 (+ 223us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.808310 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 02:30:44.808483 31218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7692e8054df40e48800faa8a49dd745" candidate_uuid: "9770676d6b4d404382235e92dca32f59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c33e9822ee23478bb9d862962c625412" is_pre_election: true
I20260925 02:30:44.808617 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.805543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:34033 (local address 127.11.220.194:53583)
0925 02:30:44.805859 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:44.805870 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:44.805952 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:44.806081 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:44.806082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:44.806084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:44.806148 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.806150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.806803 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.806804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.807517 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:44.807519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.808063 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.808064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:44.808164 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.808169 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 02:30:44.808479 (+ 310us) client_negotiation.cc:240] Negotiation successful
0925 02:30:44.808532 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":154,"thread_start_us":97,"threads_started":1}
I20260925 02:30:44.808640 31218 raft_consensus.cc:2454] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9770676d6b4d404382235e92dca32f59 in term 0.
I20260925 02:30:44.808887 31287 leader_election.cc:302] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9770676d6b4d404382235e92dca32f59, c33e9822ee23478bb9d862962c625412; no voters:
I20260925 02:30:44.808903 31486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7692e8054df40e48800faa8a49dd745" candidate_uuid: "9770676d6b4d404382235e92dca32f59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4631b7592df47b99deedb78c0c345d6" is_pre_election: true
I20260925 02:30:44.809018 31486 raft_consensus.cc:2454] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9770676d6b4d404382235e92dca32f59 in term 0.
I20260925 02:30:44.809026 31553 raft_consensus.cc:2781] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:30:44.809092 31553 raft_consensus.cc:482] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:30:44.809121 31553 raft_consensus.cc:3037] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:44.809798 31553 raft_consensus.cc:504] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 } } peers { permanent_uuid: "9770676d6b4d404382235e92dca32f59" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 43459 } } peers { permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 } }
I20260925 02:30:44.809934 31553 leader_election.cc:288] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [CANDIDATE]: Term 1 election: Requested vote from peers c4631b7592df47b99deedb78c0c345d6 (127.11.220.195:34033), c33e9822ee23478bb9d862962c625412 (127.11.220.193:40345)
I20260925 02:30:44.810109 31218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7692e8054df40e48800faa8a49dd745" candidate_uuid: "9770676d6b4d404382235e92dca32f59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c33e9822ee23478bb9d862962c625412"
I20260925 02:30:44.810123 31486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7692e8054df40e48800faa8a49dd745" candidate_uuid: "9770676d6b4d404382235e92dca32f59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4631b7592df47b99deedb78c0c345d6"
I20260925 02:30:44.810189 31486 raft_consensus.cc:3037] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:44.810194 31218 raft_consensus.cc:3037] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:44.810860 31218 raft_consensus.cc:2454] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9770676d6b4d404382235e92dca32f59 in term 1.
I20260925 02:30:44.810913 31486 raft_consensus.cc:2454] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9770676d6b4d404382235e92dca32f59 in term 1.
I20260925 02:30:44.811053 31287 leader_election.cc:302] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9770676d6b4d404382235e92dca32f59, c33e9822ee23478bb9d862962c625412; no voters:
I20260925 02:30:44.811167 31553 raft_consensus.cc:2781] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:44.811293 31553 raft_consensus.cc:686] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 1 LEADER]: Becoming Leader. State: Replica: 9770676d6b4d404382235e92dca32f59, State: Running, Role: LEADER
I20260925 02:30:44.811383 31553 consensus_queue.cc:237] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 } } peers { permanent_uuid: "9770676d6b4d404382235e92dca32f59" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 43459 } } peers { permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 } }
I20260925 02:30:44.812142 31073 catalog_manager.cc:5495] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9770676d6b4d404382235e92dca32f59 (127.11.220.194). New cstate: current_term: 1 leader_uuid: "9770676d6b4d404382235e92dca32f59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9770676d6b4d404382235e92dca32f59" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 } health_report { overall_health: UNKNOWN } } }
W20260925 02:30:44.823839 31266 tablet.cc:2369] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 02:30:44.866055 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.863901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.11.220.194:43459)
0925 02:30:44.863946 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:44.863947 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:44.863956 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:30:44.864027 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:44.864027 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:44.864046 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:44.864117 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:30:44.864247 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.864683 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.865329 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:44.865509 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:44.865573 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:44.865705 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:30:44.865811 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:30:44.865908 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:44.865953 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 02:30:44.865961 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 02:30:44.867617 31218 raft_consensus.cc:1261] T a7692e8054df40e48800faa8a49dd745 P c33e9822ee23478bb9d862962c625412 [term 1 FOLLOWER]: Refusing update from remote peer 9770676d6b4d404382235e92dca32f59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:44.867620 31486 raft_consensus.cc:1261] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6 [term 1 FOLLOWER]: Refusing update from remote peer 9770676d6b4d404382235e92dca32f59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:44.867918 31553 consensus_queue.cc:1035] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c33e9822ee23478bb9d862962c625412" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 40345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:44.868063 31557 consensus_queue.cc:1035] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4631b7592df47b99deedb78c0c345d6" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:44.870313 31561 mvcc.cc:205] Tried to move back new op lower bound from 7333082910175481856 to 7333082909947940864. Current Snapshot: MvccSnapshot[applied={T|T < 7333082910175481856}]
I20260925 02:30:44.872072 31560 mvcc.cc:205] Tried to move back new op lower bound from 7333082910175481856 to 7333082909947940864. Current Snapshot: MvccSnapshot[applied={T|T < 7333082910175481856 or (T in {7333082910175481856})}]
I20260925 02:30:44.872392 31072 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:44.872499 31072 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:44.873230 31072 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fb1a81d337440babc208a5f61017b92: Sending DeleteTablet for 3 replicas of tablet a7692e8054df40e48800faa8a49dd745
I20260925 02:30:44.873481 31466 tablet_service.cc:1514] Processing DeleteTablet for tablet a7692e8054df40e48800faa8a49dd745 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:44 UTC) from {username='slave'} at 127.0.0.1:48582
I20260925 02:30:44.873488 31332 tablet_service.cc:1514] Processing DeleteTablet for tablet a7692e8054df40e48800faa8a49dd745 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:44 UTC) from {username='slave'} at 127.0.0.1:32978
I20260925 02:30:44.873525 31198 tablet_service.cc:1514] Processing DeleteTablet for tablet a7692e8054df40e48800faa8a49dd745 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:44 UTC) from {username='slave'} at 127.0.0.1:46676
I20260925 02:30:44.873705 31571 tablet_replica.cc:330] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: stopping tablet replica
I20260925 02:30:44.873853 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 31135
I20260925 02:30:44.876425 31570 tablet_replica.cc:330] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: stopping tablet replica
I20260925 02:30:44.876531 31570 raft_consensus.cc:2229] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:30:44.876612 31570 raft_consensus.cc:2258] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:44.876962 31570 ts_tablet_manager.cc:1905] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:44.877166 31571 raft_consensus.cc:2229] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:44.877269 31571 raft_consensus.cc:2258] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:44.877647 31571 ts_tablet_manager.cc:1905] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:44.878662 31570 ts_tablet_manager.cc:1918] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:44.878738 31570 log.cc:1196] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/a7692e8054df40e48800faa8a49dd745
I20260925 02:30:44.879007 31570 ts_tablet_manager.cc:1939] T a7692e8054df40e48800faa8a49dd745 P 9770676d6b4d404382235e92dca32f59: Deleting consensus metadata
I20260925 02:30:44.879376 31056 catalog_manager.cc:4841] TS 9770676d6b4d404382235e92dca32f59 (127.11.220.194:43459): tablet a7692e8054df40e48800faa8a49dd745 (table test-table [id=efa40a34f8d642d793386aac81be9162]) successfully deleted
I20260925 02:30:44.879518 31571 ts_tablet_manager.cc:1918] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:44.879587 31571 log.cc:1196] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/a7692e8054df40e48800faa8a49dd745
I20260925 02:30:44.879827 31571 ts_tablet_manager.cc:1939] T a7692e8054df40e48800faa8a49dd745 P c4631b7592df47b99deedb78c0c345d6: Deleting consensus metadata
I20260925 02:30:44.880208 31060 catalog_manager.cc:4841] TS c4631b7592df47b99deedb78c0c345d6 (127.11.220.195:34033): tablet a7692e8054df40e48800faa8a49dd745 (table test-table [id=efa40a34f8d642d793386aac81be9162]) successfully deleted
W20260925 02:30:44.881534 31058 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:40345 (error 108)
I20260925 02:30:44.881752 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 31269
I20260925 02:30:44.882344 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:44.881819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:40345 (local address 127.0.0.1:46692)
0925 02:30:44.882265 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:44.882310 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:40345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424}
W20260925 02:30:44.882480 31058 catalog_manager.cc:4584] TS c33e9822ee23478bb9d862962c625412 (127.11.220.193:40345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7692e8054df40e48800faa8a49dd745: Network error: Client connection negotiation failed: client connection to 127.11.220.193:40345: connect: Connection refused (error 111)
I20260925 02:30:44.888250 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 31402
I20260925 02:30:44.892843 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 31041
2026-09-25T02:30:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (844 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:44 dist-test-slave-dn4n krb5kdc[31576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:44 dist-test-slave-dn4n krb5kdc[31576](info): set up 2 sockets
Sep 25 02:30:44 dist-test-slave-dn4n krb5kdc[31576](info): commencing operation
krb5kdc: starting...
W20260925 02:30:45.861322 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:45 dist-test-slave-dn4n krb5kdc[31576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:30:45.963052 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:44121
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:44121
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:46.035674 31589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:46.035840 31589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:46.035858 31589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:46.036882 31589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:46.036916 31589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:46.036928 31589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:46.036940 31589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:46.036952 31589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:46.038230 31589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:44121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:44121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:46.038391 31589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:46.038522 31589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:46.042925 31599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:46.042938 31589 server_base.cc:1034] running on GCE node
W20260925 02:30:46.042900 31597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:46.042892 31596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:46.043499 31589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:46.043766 31589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:46.044907 31589 hybrid_clock.cc:648] HybridClock initialized: now 1790303446044902 us; error 33 us; skew 500 ppm
Sep 25 02:30:46 dist-test-slave-dn4n krb5kdc[31576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:46.047148 31589 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:30:46.047451 31589 webserver.cc:460] Webserver started at http://127.11.220.254:42209/ using document root <none> and password file <none>
I20260925 02:30:46.047617 31589 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:46.047650 31589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:46.047721 31589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:46.048704 31589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "55d812e4680047238be5a471e128a1fc"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "5f62254eaa15ab57876bbc2f911ee087"
server_key_iv: "cd8c8c1e76eac37c123bc3480d581715"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.048998 31589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "55d812e4680047238be5a471e128a1fc"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "5f62254eaa15ab57876bbc2f911ee087"
server_key_iv: "cd8c8c1e76eac37c123bc3480d581715"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.050081 31589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:46.050629 31606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.050771 31589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:46.050836 31589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "55d812e4680047238be5a471e128a1fc"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "5f62254eaa15ab57876bbc2f911ee087"
server_key_iv: "cd8c8c1e76eac37c123bc3480d581715"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.050911 31589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:46.077849 31589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:46.079367 31589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:46.079499 31589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:46.083046 31589 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:44121
I20260925 02:30:46.083076 31658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:44121 every 8 connection(s)
I20260925 02:30:46.083375 31589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:46.084103 31659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:46.086015 31659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc: Bootstrap starting.
I20260925 02:30:46.086576 31659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:46.086886 31659 log.cc:824] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:46.087348 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 31589
I20260925 02:30:46.087459 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:46.087582 31659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc: No bootstrap required, opened a new log
I20260925 02:30:46.087656 12147 external_mini_cluster.cc:1428] Setting key 75480f64803f817dad419605bb34caad
I20260925 02:30:46.089757 31659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d812e4680047238be5a471e128a1fc" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44121 } }
I20260925 02:30:46.089871 31659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:46.089892 31659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55d812e4680047238be5a471e128a1fc, State: Initialized, Role: FOLLOWER
I20260925 02:30:46.090018 31659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d812e4680047238be5a471e128a1fc" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44121 } }
I20260925 02:30:46.090075 31659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:46.090094 31659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:46.090155 31659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:46.090641 31659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d812e4680047238be5a471e128a1fc" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44121 } }
I20260925 02:30:46.090741 31659 leader_election.cc:302] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55d812e4680047238be5a471e128a1fc; no voters:
I20260925 02:30:46.090916 31659 leader_election.cc:288] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:46.091084 31667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:46.091151 31659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:46.091221 31667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [term 1 LEADER]: Becoming Leader. State: Replica: 55d812e4680047238be5a471e128a1fc, State: Running, Role: LEADER
I20260925 02:30:46.091302 31667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d812e4680047238be5a471e128a1fc" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44121 } }
I20260925 02:30:46.091625 31667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55d812e4680047238be5a471e128a1fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d812e4680047238be5a471e128a1fc" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44121 } } }
I20260925 02:30:46.091756 31667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:46.091899 31666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55d812e4680047238be5a471e128a1fc. Latest consensus state: current_term: 1 leader_uuid: "55d812e4680047238be5a471e128a1fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d812e4680047238be5a471e128a1fc" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 44121 } } }
I20260925 02:30:46.092018 31666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:46.092162 31675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:46.092692 31675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:46.094296 31675 catalog_manager.cc:1347] Generated new cluster ID: 7c62fa95087e476fb05b120943af4a1c
I20260925 02:30:46.094336 31675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:30:46 dist-test-slave-dn4n krb5kdc[31576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:46.098395 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.088100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.11.220.254:44121)
0925 02:30:46.088270 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.088277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.088307 (+ 30us) server_negotiation.cc:407] Connection header received
0925 02:30:46.088813 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.088819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.088859 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.088955 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:46.089663 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.090844 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.091846 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.092439 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.092659 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.095182 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:46.095185 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:46.095185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:46.095196 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:46.096593 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:46.096940 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:46.096943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:46.096943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:46.097025 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:46.097205 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:46.097206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:46.097206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:46.097449 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:46.097600 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.098089 (+ 489us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.098216 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":103,"threads_started":1}
I20260925 02:30:46.109910 31675 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:46.110455 31675 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:46.117503 31675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc: Generated new TSK 0
I20260925 02:30:46.117654 31675 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.220.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
Entry for principal kudu/127.11.220.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab.
I20260925 02:30:46.149158 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44121
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:46.223558 31687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:46.223722 31687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:46.223742 31687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:46.224766 31687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:46.224798 31687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:46.224823 31687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:30:46.226063 31687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.193.keytab
--principal=kudu/127.11.220.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:46.226205 31687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:46.226363 31687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:46.230643 31693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:46.230650 31695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:46.230652 31692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:46.230729 31687 server_base.cc:1034] running on GCE node
I20260925 02:30:46.230993 31687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:46.231213 31687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:46.232348 31687 hybrid_clock.cc:648] HybridClock initialized: now 1790303446232333 us; error 30 us; skew 500 ppm
Sep 25 02:30:46 dist-test-slave-dn4n krb5kdc[31576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:46.234481 31687 init.cc:377] Logged in from keytab as kudu/127.11.220.193@KRBTEST.COM (short username kudu)
I20260925 02:30:46.234967 31687 webserver.cc:460] Webserver started at http://127.11.220.193:39277/ using document root <none> and password file <none>
I20260925 02:30:46.235237 31687 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:46.235325 31687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:46.235466 31687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:46.236523 31687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "524f1741c6dd4f91b443de5da4f897c8"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "6ac77b0cbecb2db9b47b2a2dcb5d5399"
server_key_iv: "3e0bb44de5efd60d7d7f7a399ca0e59d"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.236801 31687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "524f1741c6dd4f91b443de5da4f897c8"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "6ac77b0cbecb2db9b47b2a2dcb5d5399"
server_key_iv: "3e0bb44de5efd60d7d7f7a399ca0e59d"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.237965 31687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:30:46.238803 31702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.238987 31687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:30:46.239055 31687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "524f1741c6dd4f91b443de5da4f897c8"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "6ac77b0cbecb2db9b47b2a2dcb5d5399"
server_key_iv: "3e0bb44de5efd60d7d7f7a399ca0e59d"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.239143 31687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:46.258531 31687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:46.260169 31687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:46.260318 31687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:46.260615 31687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:46.260916 31687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:46.260948 31687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.260967 31687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:46.261013 31687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.266425 31687 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:38479
I20260925 02:30:46.266476 31818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:38479 every 8 connection(s)
I20260925 02:30:46.266737 31687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
Sep 25 02:30:46 dist-test-slave-dn4n krb5kdc[31576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.193@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:46.273064 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 31687
I20260925 02:30:46.273173 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:30:46.273386 12147 external_mini_cluster.cc:1428] Setting key 40ed512694e107939e510007e17779b3
I20260925 02:30:46.275010 31821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.267322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:44121 (local address 127.11.220.193:43069)
0925 02:30:46.267450 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:46.267485 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:46.267888 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:46.268071 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:46.268074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:46.268594 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:46.268761 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.268764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.269438 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.269438 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.270137 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.270139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.270456 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.270456 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.270545 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.270920 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:46.270927 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:46.272122 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:46.273786 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:46.273792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:46.273795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:46.274099 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:46.274293 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:46.274294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:46.274294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:46.274379 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:46.274619 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:46.274623 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:46.274715 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 02:30:46.274784 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:30:46.274879 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":68,"threads_started":1}
I20260925 02:30:46.275034 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.267345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:43069 (local address 127.11.220.254:44121)
0925 02:30:46.267384 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.267386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.267563 (+ 177us) server_negotiation.cc:407] Connection header received
0925 02:30:46.267958 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.267959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.267968 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.268025 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:46.268829 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.269354 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.270205 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.270388 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.270435 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.272198 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:46.272200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:46.272200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:46.272205 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:46.273711 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:46.274170 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:46.274171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:46.274171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:46.274230 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:46.274433 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:46.274434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:46.274434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:46.274561 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:46.274625 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.274821 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.274901 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:30:46.275729 31819 heartbeater.cc:347] Connected to a master server at 127.11.220.254:44121
I20260925 02:30:46.275806 31819 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:46.275965 31819 heartbeater.cc:511] Master 127.11.220.254:44121 requested a full tablet report, sending...
I20260925 02:30:46.276410 31623 ts_manager.cc:194] Registered new tserver with Master: 524f1741c6dd4f91b443de5da4f897c8 (127.11.220.193:38479)
I20260925 02:30:46.276962 31623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.193@KRBTEST.COM'} at 127.11.220.193:43069
WARNING: no policy specified for kudu/127.11.220.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
Entry for principal kudu/127.11.220.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab.
I20260925 02:30:46.295755 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44121
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:46.367010 31824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:46.367158 31824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:46.367177 31824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:46.368201 31824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:46.368237 31824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:46.368290 31824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:30:46.369647 31824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.194.keytab
--principal=kudu/127.11.220.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:46.369871 31824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:46.370078 31824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:46.374531 31833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:46.374598 31831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:46.374589 31830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:46.374609 31824 server_base.cc:1034] running on GCE node
I20260925 02:30:46.374826 31824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:46.375036 31824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:46.376189 31824 hybrid_clock.cc:648] HybridClock initialized: now 1790303446376184 us; error 31 us; skew 500 ppm
Sep 25 02:30:46 dist-test-slave-dn4n krb5kdc[31576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:46.378527 31824 init.cc:377] Logged in from keytab as kudu/127.11.220.194@KRBTEST.COM (short username kudu)
I20260925 02:30:46.378913 31824 webserver.cc:460] Webserver started at http://127.11.220.194:36607/ using document root <none> and password file <none>
I20260925 02:30:46.379144 31824 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:46.379206 31824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:46.379324 31824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:46.380227 31824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "eb4ce91d649c4430aeae613c46b6391a"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "3b6e6f588f73796c78f8c0307906ed5e"
server_key_iv: "0ff076797559a7af6422a1ce3c735acb"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.380590 31824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "eb4ce91d649c4430aeae613c46b6391a"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "3b6e6f588f73796c78f8c0307906ed5e"
server_key_iv: "0ff076797559a7af6422a1ce3c735acb"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.381739 31824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:30:46.382414 31840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.382580 31824 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:46.382670 31824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "eb4ce91d649c4430aeae613c46b6391a"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "3b6e6f588f73796c78f8c0307906ed5e"
server_key_iv: "0ff076797559a7af6422a1ce3c735acb"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.382751 31824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:46.404554 31824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:46.406391 31824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:46.406528 31824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:46.406713 31824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:46.406960 31824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:46.407006 31824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.407034 31824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:46.407058 31824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.412128 31824 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:46751
I20260925 02:30:46.412177 31956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:46751 every 8 connection(s)
I20260925 02:30:46.412415 31824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
Sep 25 02:30:46 dist-test-slave-dn4n krb5kdc[31576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.194@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:46.419888 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 31824
I20260925 02:30:46.419917 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.412972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:44121 (local address 127.11.220.194:48671)
0925 02:30:46.413114 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:46.413149 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:46.413515 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:46.413681 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:46.413684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:46.414124 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:46.414271 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.414274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.414809 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.414810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.415467 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.415468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.415808 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.415809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.415909 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.416289 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:46.416296 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:46.417584 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:46.418930 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:46.418934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:46.418936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:46.419117 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:46.419268 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:46.419269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:46.419269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:46.419354 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:46.419532 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:46.419536 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:46.419629 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 02:30:46.419698 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 02:30:46.419793 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260925 02:30:46.419940 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.413002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:48671 (local address 127.11.220.254:44121)
0925 02:30:46.413041 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.413043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.413219 (+ 176us) server_negotiation.cc:407] Connection header received
0925 02:30:46.413584 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.413585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.413594 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.413636 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:46.414332 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.414742 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.415532 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.415697 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.415759 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.417664 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:46.417665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:46.417666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:46.417671 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:46.418869 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:46.419171 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:46.419171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:46.419172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:46.419215 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:46.419401 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:46.419401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:46.419401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:46.419484 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:46.419529 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.419759 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.419830 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:30:46.420010 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:30:46.420166 12147 external_mini_cluster.cc:1428] Setting key 11444572a559534652d2ea1a532cc774
I20260925 02:30:46.420624 31957 heartbeater.cc:347] Connected to a master server at 127.11.220.254:44121
I20260925 02:30:46.420706 31957 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:46.420854 31957 heartbeater.cc:511] Master 127.11.220.254:44121 requested a full tablet report, sending...
I20260925 02:30:46.421221 31622 ts_manager.cc:194] Registered new tserver with Master: eb4ce91d649c4430aeae613c46b6391a (127.11.220.194:46751)
I20260925 02:30:46.421710 31622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.194@KRBTEST.COM'} at 127.11.220.194:48671
WARNING: no policy specified for kudu/127.11.220.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
Entry for principal kudu/127.11.220.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab.
I20260925 02:30:46.441011 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44121
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:46.514384 31962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:46.514534 31962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:46.514554 31962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:46.515501 31962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:46.515528 31962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:46.515549 31962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:30:46.516829 31962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.195.keytab
--principal=kudu/127.11.220.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:44121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:46.516978 31962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:46.517184 31962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:46.521597 31967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:46.521620 31971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:46.521647 31962 server_base.cc:1034] running on GCE node
W20260925 02:30:46.521625 31968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:46.521942 31962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:46.522186 31962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:46.523350 31962 hybrid_clock.cc:648] HybridClock initialized: now 1790303446523339 us; error 26 us; skew 500 ppm
Sep 25 02:30:46 dist-test-slave-dn4n krb5kdc[31576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:46.525681 31962 init.cc:377] Logged in from keytab as kudu/127.11.220.195@KRBTEST.COM (short username kudu)
I20260925 02:30:46.526038 31962 webserver.cc:460] Webserver started at http://127.11.220.195:38355/ using document root <none> and password file <none>
I20260925 02:30:46.526197 31962 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:46.526233 31962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:46.526327 31962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:46.527182 31962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "5339570f5dc344829376d7e962f7f974"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "454f03c63430ef6c71df346ab8bc25d2"
server_key_iv: "0fadd417959ddb59a836aa3fcfc3920c"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.527436 31962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "5339570f5dc344829376d7e962f7f974"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "454f03c63430ef6c71df346ab8bc25d2"
server_key_iv: "0fadd417959ddb59a836aa3fcfc3920c"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.528553 31962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:30:46.529152 31978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.529295 31962 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:46.529369 31962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "5339570f5dc344829376d7e962f7f974"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "454f03c63430ef6c71df346ab8bc25d2"
server_key_iv: "0fadd417959ddb59a836aa3fcfc3920c"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.529433 31962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:46.536757 31962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:46.538703 31962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:46.538796 31962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:46.538978 31962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:46.539243 31962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:46.539290 31962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.539327 31962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:46.539352 31962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.544481 31962 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:42857
I20260925 02:30:46.544533 32093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:42857 every 8 connection(s)
I20260925 02:30:46.544791 31962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
Sep 25 02:30:46 dist-test-slave-dn4n krb5kdc[31576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.195@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:30:46.552112 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.545424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:50311 (local address 127.11.220.254:44121)
0925 02:30:46.545488 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.545490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.545682 (+ 192us) server_negotiation.cc:407] Connection header received
0925 02:30:46.546036 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.546037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.546046 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.546089 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:46.546782 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.547200 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.548021 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.548188 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.548247 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.549902 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:46.549903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:46.549904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:46.549909 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:46.551138 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:46.551420 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:46.551420 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:46.551421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:46.551462 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:46.551631 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:46.551632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:46.551632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:46.551711 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:46.551752 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.551984 (+ 232us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.552047 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 02:30:46.552253 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.545396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:44121 (local address 127.11.220.195:50311)
0925 02:30:46.545549 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:46.545597 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:46.545964 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:46.546131 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:46.546134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:46.546571 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:46.546719 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.546722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.547267 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.547268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.547939 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.547941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.548265 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.548265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.548346 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.548718 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:30:46.548724 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:46.549828 (+ 1104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:46.551194 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:46.551199 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:46.551200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:46.551365 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:46.551510 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:30:46.551511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:30:46.551511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:30:46.551586 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:30:46.551757 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:46.551761 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:46.551863 (+ 102us) client_negotiation.cc:769] Sending connection context
0925 02:30:46.551938 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 02:30:46.552053 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260925 02:30:46.552760 32094 heartbeater.cc:347] Connected to a master server at 127.11.220.254:44121
I20260925 02:30:46.552856 32094 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:46.553016 32094 heartbeater.cc:511] Master 127.11.220.254:44121 requested a full tablet report, sending...
I20260925 02:30:46.553418 31622 ts_manager.cc:194] Registered new tserver with Master: 5339570f5dc344829376d7e962f7f974 (127.11.220.195:42857)
I20260925 02:30:46.553781 31622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.220.195@KRBTEST.COM'} at 127.11.220.195:50311
I20260925 02:30:46.554703 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 31962
I20260925 02:30:46.554780 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:30:46.554900 12147 external_mini_cluster.cc:1428] Setting key 6f6529ec1e1ac5465bf51e4092960ff8
I20260925 02:30:46.555617 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:30:46.561879 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.556806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40558 (local address 127.11.220.254:44121)
0925 02:30:46.556871 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.556873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.556957 (+ 84us) server_negotiation.cc:407] Connection header received
0925 02:30:46.557021 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.557022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.557031 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.557072 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:46.557538 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.557941 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.558615 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.558776 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.558840 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.559757 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:46.559758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:46.559759 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:46.559765 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:46.560849 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:46.561198 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:46.561199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:46.561199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:46.561248 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:46.561440 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:46.561440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:46.561441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:46.561556 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:46.561651 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.561740 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.561799 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 02:30:46.563210 31622 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:30:46.564778 31622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:30:46.570613 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.566595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:46751 (local address 127.0.0.1:53186)
0925 02:30:46.566716 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:46.566751 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:46.566808 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:46.567225 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:46.567227 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:46.567229 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:46.567283 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.567284 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.568603 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.568604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.569646 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.569648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.570267 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.570268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.570375 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.570387 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 02:30:46.570492 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 02:30:46.570550 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":46,"threads_started":1}
I20260925 02:30:46.570624 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.566877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53186 (local address 127.11.220.194:46751)
0925 02:30:46.567062 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.567066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.567076 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:30:46.567115 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.567117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.567131 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.567219 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:46.567336 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.568508 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.569727 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.570179 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.570267 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.570325 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.570511 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.570549 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260925 02:30:46.570851 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.566598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:38479 (local address 127.0.0.1:45442)
0925 02:30:46.566723 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:46.566753 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:46.566808 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:46.567233 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:46.567234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:46.567236 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:46.567308 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.567310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.568290 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.568291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.569599 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.569602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.570603 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.570604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.570669 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.570675 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 02:30:46.570725 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 02:30:46.570776 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":34,"thread_start_us":62,"threads_started":1}
I20260925 02:30:46.571038 32105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.566772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:42857 (local address 127.0.0.1:56844)
0925 02:30:46.567321 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:46.567330 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:46.567379 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:46.567619 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:46.567619 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:46.567622 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:46.567661 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.567663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.568811 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.568812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.569710 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.569713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.570601 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.570602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.570669 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.570677 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 02:30:46.570725 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 02:30:46.570776 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":16,"thread_start_us":72,"threads_started":1}
I20260925 02:30:46.571094 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.566698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45442 (local address 127.11.220.193:38479)
0925 02:30:46.567089 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.567093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.567101 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:30:46.567140 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.567141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.567151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.567234 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:46.567390 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.568192 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.569683 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.570429 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.570515 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.570562 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.570956 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.571004 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":85,"threads_started":1}
I20260925 02:30:46.571178 32108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.566952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56844 (local address 127.11.220.195:42857)
0925 02:30:46.567106 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.567110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.567375 (+ 265us) server_negotiation.cc:407] Connection header received
0925 02:30:46.567464 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.567466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.567479 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.567551 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:46.567735 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.568718 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.569796 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.570434 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.570514 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.570562 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.570816 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.570851 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
I20260925 02:30:46.571339 31889 tablet_service.cc:1467] Processing CreateTablet for tablet a8b9c1f4659c4c58bc24dd3b0ffb870a (DEFAULT_TABLE table=test-table [id=fcb4040076de4704a43cbff4595e063e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:46.571676 31889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8b9c1f4659c4c58bc24dd3b0ffb870a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:46.571727 31751 tablet_service.cc:1467] Processing CreateTablet for tablet a8b9c1f4659c4c58bc24dd3b0ffb870a (DEFAULT_TABLE table=test-table [id=fcb4040076de4704a43cbff4595e063e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:46.572021 31751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8b9c1f4659c4c58bc24dd3b0ffb870a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:46.573040 32027 tablet_service.cc:1467] Processing CreateTablet for tablet a8b9c1f4659c4c58bc24dd3b0ffb870a (DEFAULT_TABLE table=test-table [id=fcb4040076de4704a43cbff4595e063e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:46.573333 32027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8b9c1f4659c4c58bc24dd3b0ffb870a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:46.574496 32109 tablet_bootstrap.cc:492] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8: Bootstrap starting.
I20260925 02:30:46.574908 32110 tablet_bootstrap.cc:492] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: Bootstrap starting.
I20260925 02:30:46.575016 32109 tablet_bootstrap.cc:654] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:46.575374 32109 log.cc:824] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:46.575464 32110 tablet_bootstrap.cc:654] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:46.576002 32109 tablet_bootstrap.cc:492] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8: No bootstrap required, opened a new log
I20260925 02:30:46.576040 32109 ts_tablet_manager.cc:1397] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:46.576013 32110 log.cc:824] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:46.576431 32113 tablet_bootstrap.cc:492] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: Bootstrap starting.
I20260925 02:30:46.576718 32110 tablet_bootstrap.cc:492] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: No bootstrap required, opened a new log
I20260925 02:30:46.576771 32110 ts_tablet_manager.cc:1397] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:46.577032 32113 tablet_bootstrap.cc:654] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:46.577471 32113 log.cc:824] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:46.577873 32109 raft_consensus.cc:348] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 } } peers { permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 } } peers { permanent_uuid: "5339570f5dc344829376d7e962f7f974" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 42857 } }
I20260925 02:30:46.578056 32109 raft_consensus.cc:374] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:46.578099 32113 tablet_bootstrap.cc:492] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: No bootstrap required, opened a new log
I20260925 02:30:46.578126 32109 raft_consensus.cc:729] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524f1741c6dd4f91b443de5da4f897c8, State: Initialized, Role: FOLLOWER
I20260925 02:30:46.578186 32113 ts_tablet_manager.cc:1397] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 02:30:46.578255 32109 consensus_queue.cc:260] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 } } peers { permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 } } peers { permanent_uuid: "5339570f5dc344829376d7e962f7f974" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 42857 } }
I20260925 02:30:46.578516 32109 ts_tablet_manager.cc:1428] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:30:46.578544 31819 heartbeater.cc:503] Master 127.11.220.254:44121 was elected leader, sending a full tablet report...
I20260925 02:30:46.578574 32110 raft_consensus.cc:348] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 } } peers { permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 } } peers { permanent_uuid: "5339570f5dc344829376d7e962f7f974" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 42857 } }
I20260925 02:30:46.578706 32110 raft_consensus.cc:374] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:46.578727 32110 raft_consensus.cc:729] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb4ce91d649c4430aeae613c46b6391a, State: Initialized, Role: FOLLOWER
I20260925 02:30:46.578824 32110 consensus_queue.cc:260] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 } } peers { permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 } } peers { permanent_uuid: "5339570f5dc344829376d7e962f7f974" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 42857 } }
I20260925 02:30:46.579056 32110 ts_tablet_manager.cc:1428] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:30:46.579071 31957 heartbeater.cc:503] Master 127.11.220.254:44121 was elected leader, sending a full tablet report...
I20260925 02:30:46.580108 32113 raft_consensus.cc:348] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 } } peers { permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 } } peers { permanent_uuid: "5339570f5dc344829376d7e962f7f974" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 42857 } }
I20260925 02:30:46.580262 32113 raft_consensus.cc:374] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:46.580305 32113 raft_consensus.cc:729] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5339570f5dc344829376d7e962f7f974, State: Initialized, Role: FOLLOWER
I20260925 02:30:46.580395 32113 consensus_queue.cc:260] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 } } peers { permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 } } peers { permanent_uuid: "5339570f5dc344829376d7e962f7f974" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 42857 } }
I20260925 02:30:46.580636 32113 ts_tablet_manager.cc:1428] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:30:46.580680 32094 heartbeater.cc:503] Master 127.11.220.254:44121 was elected leader, sending a full tablet report...
I20260925 02:30:46.632740 32117 raft_consensus.cc:482] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:46.632830 32117 raft_consensus.cc:504] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 } } peers { permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 } } peers { permanent_uuid: "5339570f5dc344829376d7e962f7f974" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 42857 } }
I20260925 02:30:46.633134 32117 leader_election.cc:288] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 524f1741c6dd4f91b443de5da4f897c8 (127.11.220.193:38479), eb4ce91d649c4430aeae613c46b6391a (127.11.220.194:46751)
I20260925 02:30:46.635777 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.633347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:33623 (local address 127.11.220.193:38479)
0925 02:30:46.633403 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.633405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.633413 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:30:46.633445 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.633446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.633452 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.633515 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:46.633661 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.634132 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.635101 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.635452 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.635529 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.635538 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.635684 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.635720 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:30:46.635777 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.633261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:38479 (local address 127.11.220.195:33623)
0925 02:30:46.633323 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:46.633335 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:46.633371 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:46.633517 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:46.633518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:46.633520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:46.633593 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.633594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.634216 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.634217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.634983 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.634985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.635534 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.635535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.635610 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.635613 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:46.635678 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:30:46.635713 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":49}
I20260925 02:30:46.635905 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.633343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:47745 (local address 127.11.220.194:46751)
0925 02:30:46.633419 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.633421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.633527 (+ 106us) server_negotiation.cc:407] Connection header received
0925 02:30:46.633591 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.633593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.633601 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.633666 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:46.633811 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.634272 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.635203 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.635579 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.635669 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.635677 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.635815 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.635850 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 02:30:46.635921 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.633296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:46751 (local address 127.11.220.195:47745)
0925 02:30:46.633464 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:46.633471 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:46.633542 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:46.633672 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:46.633673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:46.633675 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:46.633742 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.633755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.634351 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.634353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.635109 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.635111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.635670 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.635671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.635738 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.635741 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:46.635806 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:30:46.635842 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260925 02:30:46.636109 31772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b9c1f4659c4c58bc24dd3b0ffb870a" candidate_uuid: "5339570f5dc344829376d7e962f7f974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "524f1741c6dd4f91b443de5da4f897c8" is_pre_election: true
I20260925 02:30:46.636170 31910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b9c1f4659c4c58bc24dd3b0ffb870a" candidate_uuid: "5339570f5dc344829376d7e962f7f974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb4ce91d649c4430aeae613c46b6391a" is_pre_election: true
I20260925 02:30:46.636236 31772 raft_consensus.cc:2454] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5339570f5dc344829376d7e962f7f974 in term 0.
I20260925 02:30:46.636314 31910 raft_consensus.cc:2454] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5339570f5dc344829376d7e962f7f974 in term 0.
I20260925 02:30:46.636468 31982 leader_election.cc:302] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524f1741c6dd4f91b443de5da4f897c8, 5339570f5dc344829376d7e962f7f974; no voters:
I20260925 02:30:46.636582 32117 raft_consensus.cc:2781] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:30:46.636632 32117 raft_consensus.cc:482] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:30:46.636660 32117 raft_consensus.cc:3037] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:46.637300 32117 raft_consensus.cc:504] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 } } peers { permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 } } peers { permanent_uuid: "5339570f5dc344829376d7e962f7f974" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 42857 } }
I20260925 02:30:46.637434 32117 leader_election.cc:288] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [CANDIDATE]: Term 1 election: Requested vote from peers 524f1741c6dd4f91b443de5da4f897c8 (127.11.220.193:38479), eb4ce91d649c4430aeae613c46b6391a (127.11.220.194:46751)
I20260925 02:30:46.637550 31772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b9c1f4659c4c58bc24dd3b0ffb870a" candidate_uuid: "5339570f5dc344829376d7e962f7f974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "524f1741c6dd4f91b443de5da4f897c8"
I20260925 02:30:46.637558 31910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8b9c1f4659c4c58bc24dd3b0ffb870a" candidate_uuid: "5339570f5dc344829376d7e962f7f974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb4ce91d649c4430aeae613c46b6391a"
I20260925 02:30:46.637642 31772 raft_consensus.cc:3037] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:46.637661 31910 raft_consensus.cc:3037] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:46.638347 31772 raft_consensus.cc:2454] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5339570f5dc344829376d7e962f7f974 in term 1.
I20260925 02:30:46.638429 31910 raft_consensus.cc:2454] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5339570f5dc344829376d7e962f7f974 in term 1.
I20260925 02:30:46.638515 31982 leader_election.cc:302] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524f1741c6dd4f91b443de5da4f897c8, 5339570f5dc344829376d7e962f7f974; no voters:
I20260925 02:30:46.638628 32117 raft_consensus.cc:2781] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:46.638756 32117 raft_consensus.cc:686] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 1 LEADER]: Becoming Leader. State: Replica: 5339570f5dc344829376d7e962f7f974, State: Running, Role: LEADER
I20260925 02:30:46.638839 32117 consensus_queue.cc:237] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 } } peers { permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 } } peers { permanent_uuid: "5339570f5dc344829376d7e962f7f974" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 42857 } }
I20260925 02:30:46.639597 31622 catalog_manager.cc:5495] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5339570f5dc344829376d7e962f7f974 (127.11.220.195). New cstate: current_term: 1 leader_uuid: "5339570f5dc344829376d7e962f7f974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5339570f5dc344829376d7e962f7f974" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 42857 } health_report { overall_health: HEALTHY } } }
I20260925 02:30:46.644344 32108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.642117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.11.220.195:42857)
0925 02:30:46.642147 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.642149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.642200 (+ 51us) server_negotiation.cc:407] Connection header received
0925 02:30:46.642273 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.642274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.642294 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.642359 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:30:46.642450 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.642940 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.643668 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.643851 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.643915 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.644071 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:30:46.644153 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:30:46.644237 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.644254 (+ 17us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.644270 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 02:30:46.645673 31910 raft_consensus.cc:1261] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a [term 1 FOLLOWER]: Refusing update from remote peer 5339570f5dc344829376d7e962f7f974: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:46.645673 31772 raft_consensus.cc:1261] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 524f1741c6dd4f91b443de5da4f897c8 [term 1 FOLLOWER]: Refusing update from remote peer 5339570f5dc344829376d7e962f7f974: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:46.645944 32117 consensus_queue.cc:1035] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb4ce91d649c4430aeae613c46b6391a" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:46.646098 32119 consensus_queue.cc:1035] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "524f1741c6dd4f91b443de5da4f897c8" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 38479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:46.647964 32122 mvcc.cc:205] Tried to move back new op lower bound from 7333082917458841600 to 7333082917433286656. Current Snapshot: MvccSnapshot[applied={T|T < 7333082917458841600}]
I20260925 02:30:46.650861 31623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:46.650942 31623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:46.651577 31623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55d812e4680047238be5a471e128a1fc: Sending DeleteTablet for 3 replicas of tablet a8b9c1f4659c4c58bc24dd3b0ffb870a
I20260925 02:30:46.651857 31889 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b9c1f4659c4c58bc24dd3b0ffb870a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:46 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:53186
I20260925 02:30:46.651886 31751 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b9c1f4659c4c58bc24dd3b0ffb870a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:46 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:45442
I20260925 02:30:46.651921 32027 tablet_service.cc:1514] Processing DeleteTablet for tablet a8b9c1f4659c4c58bc24dd3b0ffb870a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:46 UTC) from {username='kudu', principal='kudu/127.11.220.254@KRBTEST.COM'} at 127.0.0.1:56844
I20260925 02:30:46.652124 32132 tablet_replica.cc:330] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: stopping tablet replica
I20260925 02:30:46.652221 32132 raft_consensus.cc:2229] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:46.652319 32132 raft_consensus.cc:2258] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:46.652421 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 31687
I20260925 02:30:46.653955 32132 ts_tablet_manager.cc:1905] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:46.655580 32134 tablet_replica.cc:330] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: stopping tablet replica
I20260925 02:30:46.655613 32132 ts_tablet_manager.cc:1918] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:46.655689 32132 log.cc:1196] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/a8b9c1f4659c4c58bc24dd3b0ffb870a
I20260925 02:30:46.655716 32134 raft_consensus.cc:2229] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:30:46.655836 32134 raft_consensus.cc:2258] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:46.655953 32132 ts_tablet_manager.cc:1939] T a8b9c1f4659c4c58bc24dd3b0ffb870a P eb4ce91d649c4430aeae613c46b6391a: Deleting consensus metadata
I20260925 02:30:46.656278 32134 ts_tablet_manager.cc:1905] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:46.656325 31608 catalog_manager.cc:4841] TS eb4ce91d649c4430aeae613c46b6391a (127.11.220.194:46751): tablet a8b9c1f4659c4c58bc24dd3b0ffb870a (table test-table [id=fcb4040076de4704a43cbff4595e063e]) successfully deleted
I20260925 02:30:46.657738 32134 ts_tablet_manager.cc:1918] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:46.657805 32134 log.cc:1196] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/a8b9c1f4659c4c58bc24dd3b0ffb870a
I20260925 02:30:46.658034 32134 ts_tablet_manager.cc:1939] T a8b9c1f4659c4c58bc24dd3b0ffb870a P 5339570f5dc344829376d7e962f7f974: Deleting consensus metadata
I20260925 02:30:46.658371 31610 catalog_manager.cc:4841] TS 5339570f5dc344829376d7e962f7f974 (127.11.220.195:42857): tablet a8b9c1f4659c4c58bc24dd3b0ffb870a (table test-table [id=fcb4040076de4704a43cbff4595e063e]) successfully deleted
W20260925 02:30:46.659184 31610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:38479 (error 108)
I20260925 02:30:46.659574 32105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.659471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:38479 (local address 127.0.0.1:45450)
0925 02:30:46.659510 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:46.659537 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:38479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 02:30:46.659612 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 31824
W20260925 02:30:46.659670 31610 catalog_manager.cc:4584] TS 524f1741c6dd4f91b443de5da4f897c8 (127.11.220.193:38479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8b9c1f4659c4c58bc24dd3b0ffb870a: Network error: Client connection negotiation failed: client connection to 127.11.220.193:38479: connect: Connection refused (error 111)
I20260925 02:30:46.664794 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 31962
I20260925 02:30:46.669361 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 31589
2026-09-25T02:30:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1776 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T02:30:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:46Z Disabled control of system clock
I20260925 02:30:46.694887 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:33855
--webserver_interface=127.11.220.254
--webserver_port=0
--builtin_ntp_servers=127.11.220.212:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:33855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:30:46.766844 32139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:46.767023 32139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:46.767041 32139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:46.768239 32139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:46.768288 32139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:46.768301 32139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:46.768311 32139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:46.768352 32139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:46.769570 32139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:33855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:33855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:46.769788 32139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:46.770062 32139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:46.774266 32145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:46.774269 32144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:46.774325 32139 server_base.cc:1034] running on GCE node
W20260925 02:30:46.774278 32147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:46.774744 32139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:46.775033 32139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:46.776192 32139 hybrid_clock.cc:648] HybridClock initialized: now 1790303446776178 us; error 39 us; skew 500 ppm
I20260925 02:30:46.777390 32139 webserver.cc:460] Webserver started at http://127.11.220.254:40837/ using document root <none> and password file <none>
I20260925 02:30:46.777634 32139 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:46.777702 32139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:46.777832 32139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:46.778775 32139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "1c0891296dd44306a62d58ae71ad7cb8"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "a9a1796fbf02c8fcdabcb61ce690bb1d"
server_key_iv: "fcb09acf8d3c5af2f1f9ee553e2aaefd"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.779114 32139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "1c0891296dd44306a62d58ae71ad7cb8"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "a9a1796fbf02c8fcdabcb61ce690bb1d"
server_key_iv: "fcb09acf8d3c5af2f1f9ee553e2aaefd"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.780335 32139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:30:46.781061 32153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.781236 32139 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:46.781322 32139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "1c0891296dd44306a62d58ae71ad7cb8"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "a9a1796fbf02c8fcdabcb61ce690bb1d"
server_key_iv: "fcb09acf8d3c5af2f1f9ee553e2aaefd"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.781399 32139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:46.794690 32139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:46.794927 32139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:46.795055 32139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:46.798503 32139 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:33855
I20260925 02:30:46.798542 32206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:33855 every 8 connection(s)
I20260925 02:30:46.798808 32139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:46.799650 32207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:46.801575 32207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8: Bootstrap starting.
I20260925 02:30:46.802100 32207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:46.802376 32207 log.cc:824] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:46.802990 32207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8: No bootstrap required, opened a new log
I20260925 02:30:46.804543 32207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0891296dd44306a62d58ae71ad7cb8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33855 } }
I20260925 02:30:46.804658 32207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:46.804697 32207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c0891296dd44306a62d58ae71ad7cb8, State: Initialized, Role: FOLLOWER
I20260925 02:30:46.804826 32207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0891296dd44306a62d58ae71ad7cb8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33855 } }
I20260925 02:30:46.804904 32207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:46.804951 32207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:46.804996 32207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:46.805430 32207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0891296dd44306a62d58ae71ad7cb8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33855 } }
I20260925 02:30:46.805534 32207 leader_election.cc:302] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c0891296dd44306a62d58ae71ad7cb8; no voters:
I20260925 02:30:46.805701 32207 leader_election.cc:288] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:46.805730 32210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:46.805930 32210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [term 1 LEADER]: Becoming Leader. State: Replica: 1c0891296dd44306a62d58ae71ad7cb8, State: Running, Role: LEADER
I20260925 02:30:46.805936 32207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:46.806038 32210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0891296dd44306a62d58ae71ad7cb8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33855 } }
I20260925 02:30:46.806393 32212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c0891296dd44306a62d58ae71ad7cb8. Latest consensus state: current_term: 1 leader_uuid: "1c0891296dd44306a62d58ae71ad7cb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0891296dd44306a62d58ae71ad7cb8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33855 } } }
I20260925 02:30:46.806561 32212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:46.806466 32211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c0891296dd44306a62d58ae71ad7cb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c0891296dd44306a62d58ae71ad7cb8" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 33855 } } }
I20260925 02:30:46.806730 32211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:46.807054 32221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:46.807545 32221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:46.808648 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 32139
I20260925 02:30:46.808754 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:46.808939 12147 external_mini_cluster.cc:1428] Setting key 838b53459528e2d6f0969c36ccba9137
I20260925 02:30:46.809155 32221 catalog_manager.cc:1347] Generated new cluster ID: 2de522808cb243bc8cdf4c1b8a5aca10
I20260925 02:30:46.809206 32221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:46.814297 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.809484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56378 (local address 127.11.220.254:33855)
0925 02:30:46.809646 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.809654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.809679 (+ 25us) server_negotiation.cc:407] Connection header received
0925 02:30:46.809768 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.809773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.809827 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.809943 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:46.810165 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.811347 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.812399 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.812973 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.813074 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.813710 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:46.813713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:46.813714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:46.813880 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:46.813914 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:46.813925 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:46.813980 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.814069 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.814134 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":74,"threads_started":1}
I20260925 02:30:46.824221 32221 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:46.824764 32221 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:46.833524 32221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8: Generated new TSK 0
I20260925 02:30:46.833679 32221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:46.835474 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.193:0
--local_ip_for_outbound_sockets=127.11.220.193
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33855
--builtin_ntp_servers=127.11.220.212:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:30:46.908208 32231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:46.908360 32231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:46.908378 32231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:46.909389 32231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:46.909420 32231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:46.909440 32231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.193
I20260925 02:30:46.910589 32231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.220.193
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.193
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:46.910802 32231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:46.911060 32231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:46.915517 32236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:46.915517 32239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:46.915702 32237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:46.915849 32231 server_base.cc:1034] running on GCE node
I20260925 02:30:46.916051 32231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:46.916281 32231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:46.917420 32231 hybrid_clock.cc:648] HybridClock initialized: now 1790303446917384 us; error 51 us; skew 500 ppm
I20260925 02:30:46.918573 32231 webserver.cc:460] Webserver started at http://127.11.220.193:33783/ using document root <none> and password file <none>
I20260925 02:30:46.918752 32231 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:46.918788 32231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:46.918864 32231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:46.919834 32231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/data/instance:
uuid: "f8a80c43759d4dc78c2a7b59a45d8c99"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "6410e908ef2abe7b0672a06fce02c83c"
server_key_iv: "3f7f5547e11225db67ce7b15f2c00e14"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.920120 32231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance:
uuid: "f8a80c43759d4dc78c2a7b59a45d8c99"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "6410e908ef2abe7b0672a06fce02c83c"
server_key_iv: "3f7f5547e11225db67ce7b15f2c00e14"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.921298 32231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:30:46.921860 32245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.922024 32231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:46.922086 32231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/wal
uuid: "f8a80c43759d4dc78c2a7b59a45d8c99"
format_stamp: "Formatted at 2026-09-25 02:30:46 on dist-test-slave-dn4n"
server_key: "6410e908ef2abe7b0672a06fce02c83c"
server_key_iv: "3f7f5547e11225db67ce7b15f2c00e14"
server_key_version: "encryptionkey@0"
I20260925 02:30:46.922130 32231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:46.935564 32231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:46.935766 32231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:46.935859 32231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:46.936020 32231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:46.936219 32231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:46.936244 32231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.936261 32231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:46.936271 32231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:46.941562 32231 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.193:36147
I20260925 02:30:46.941618 32362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.193:36147 every 8 connection(s)
I20260925 02:30:46.941900 32231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/data/info.pb
I20260925 02:30:46.946574 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.942563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.193:59579 (local address 127.11.220.254:33855)
0925 02:30:46.942621 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:46.942623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:46.942839 (+ 216us) server_negotiation.cc:407] Connection header received
0925 02:30:46.943308 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:46.943309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:46.943322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:46.943379 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:46.943993 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.944541 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.945391 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.945584 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.945657 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.946355 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:46.946357 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:46.946357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:46.946361 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:46.946373 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:46.946377 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:46.946415 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:46.946478 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 02:30:46.946507 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 02:30:46.946734 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:46.942537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33855 (local address 127.11.220.193:59579)
0925 02:30:46.942729 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:46.942758 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:46.943198 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:46.943435 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:46.943438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:46.943446 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:46.943901 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.943905 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.944623 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.944625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.945308 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:46.945310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:46.945669 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:46.945670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:46.945783 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:46.946228 (+ 445us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:30:46.946228 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:46.946251 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:30:46.946269 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:46.946414 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:46.946419 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:46.946421 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:30:46.946471 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 02:30:46.946554 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260925 02:30:46.947141 32363 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33855
I20260925 02:30:46.947335 32363 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:46.947510 32363 heartbeater.cc:511] Master 127.11.220.254:33855 requested a full tablet report, sending...
I20260925 02:30:46.947933 32171 ts_manager.cc:194] Registered new tserver with Master: f8a80c43759d4dc78c2a7b59a45d8c99 (127.11.220.193:36147)
I20260925 02:30:46.948490 32171 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.193:59579
I20260925 02:30:46.949924 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 32231
I20260925 02:30:46.950009 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-0/wal/instance
I20260925 02:30:46.950138 12147 external_mini_cluster.cc:1428] Setting key 4e3ac322c50094512c588a45e428e216
I20260925 02:30:46.950877 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.194:0
--local_ip_for_outbound_sockets=127.11.220.194
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33855
--builtin_ntp_servers=127.11.220.212:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 02:30:47.023253 32367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:47.023442 32367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:47.023475 32367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:47.024576 32367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:47.024626 32367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:47.024663 32367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.194
I20260925 02:30:47.025899 32367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.220.194
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.194
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:47.026127 32367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:47.026362 32367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:47.030700 32372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:47.030943 32375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:47.030843 32373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:47.031307 32367 server_base.cc:1034] running on GCE node
I20260925 02:30:47.031502 32367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:47.031724 32367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:47.032832 32367 hybrid_clock.cc:648] HybridClock initialized: now 1790303447032816 us; error 29 us; skew 500 ppm
I20260925 02:30:47.034286 32367 webserver.cc:460] Webserver started at http://127.11.220.194:46837/ using document root <none> and password file <none>
I20260925 02:30:47.034559 32367 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:47.034612 32367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:47.034729 32367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:47.035933 32367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/data/instance:
uuid: "24f69131c4334135b2a242442acac749"
format_stamp: "Formatted at 2026-09-25 02:30:47 on dist-test-slave-dn4n"
server_key: "dd61f87dad4ced8f13b82ba019e2bff1"
server_key_iv: "16451ca27321b3ddcbcce950aec2e941"
server_key_version: "encryptionkey@0"
I20260925 02:30:47.036288 32367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance:
uuid: "24f69131c4334135b2a242442acac749"
format_stamp: "Formatted at 2026-09-25 02:30:47 on dist-test-slave-dn4n"
server_key: "dd61f87dad4ced8f13b82ba019e2bff1"
server_key_iv: "16451ca27321b3ddcbcce950aec2e941"
server_key_version: "encryptionkey@0"
I20260925 02:30:47.037672 32367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:30:47.038265 32381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:47.038404 32367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:47.038458 32367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/wal
uuid: "24f69131c4334135b2a242442acac749"
format_stamp: "Formatted at 2026-09-25 02:30:47 on dist-test-slave-dn4n"
server_key: "dd61f87dad4ced8f13b82ba019e2bff1"
server_key_iv: "16451ca27321b3ddcbcce950aec2e941"
server_key_version: "encryptionkey@0"
I20260925 02:30:47.038504 32367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:47.050557 32367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:47.050733 32367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:47.050810 32367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:47.050951 32367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:47.051201 32367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:47.051227 32367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:47.051272 32367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:47.051295 32367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:47.056190 32367 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.194:39499
I20260925 02:30:47.056250 32498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.194:39499 every 8 connection(s)
I20260925 02:30:47.056612 32367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/data/info.pb
I20260925 02:30:47.057806 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 32367
I20260925 02:30:47.057919 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/wal/instance
I20260925 02:30:47.058099 12147 external_mini_cluster.cc:1428] Setting key f74bd2578766c7a53992018a33c895db
I20260925 02:30:47.058955 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.220.195:0
--local_ip_for_outbound_sockets=127.11.220.195
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33855
--builtin_ntp_servers=127.11.220.212:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 02:30:47.061888 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.057262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:56551 (local address 127.11.220.254:33855)
0925 02:30:47.057309 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:47.057312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:47.057512 (+ 200us) server_negotiation.cc:407] Connection header received
0925 02:30:47.057938 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:47.057939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:47.057952 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:47.057999 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:47.058714 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.059250 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.060592 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.060877 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.060945 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.061636 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:47.061638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:47.061638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:47.061642 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:47.061657 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:47.061661 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:47.061714 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:47.061761 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 02:30:47.061786 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:30:47.062002 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.057250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33855 (local address 127.11.220.194:56551)
0925 02:30:47.057412 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:47.057438 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:47.057833 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:47.058078 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:47.058082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:47.058092 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:47.058646 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.058652 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.059340 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.059341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.060508 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.060510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.060956 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.060957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.061053 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.061515 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:30:47.061515 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:47.061538 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:30:47.061557 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:47.061714 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:47.061716 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:47.061719 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:47.061757 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 02:30:47.061821 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260925 02:30:47.062485 32499 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33855
I20260925 02:30:47.062592 32499 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:47.062759 32499 heartbeater.cc:511] Master 127.11.220.254:33855 requested a full tablet report, sending...
I20260925 02:30:47.063153 32171 ts_manager.cc:194] Registered new tserver with Master: 24f69131c4334135b2a242442acac749 (127.11.220.194:39499)
I20260925 02:30:47.063534 32171 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.194:56551
W20260925 02:30:47.133536 32502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:47.133700 32502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:47.133718 32502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:47.134737 32502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:47.134769 32502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:47.134791 32502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.220.195
I20260925 02:30:47.135996 32502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.220.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.220.195
--webserver_port=0
--tserver_master_addrs=127.11.220.254:33855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.220.195
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:47.136147 32502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:47.136397 32502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:47.140738 32508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:47.140748 32510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:47.140748 32507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:47.141001 32502 server_base.cc:1034] running on GCE node
I20260925 02:30:47.141191 32502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:47.141368 32502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:47.142491 32502 hybrid_clock.cc:648] HybridClock initialized: now 1790303447142473 us; error 32 us; skew 500 ppm
I20260925 02:30:47.143774 32502 webserver.cc:460] Webserver started at http://127.11.220.195:34149/ using document root <none> and password file <none>
I20260925 02:30:47.143986 32502 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:47.144026 32502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:47.144163 32502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:47.145182 32502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/data/instance:
uuid: "0535c7e43b6744dd88a10837c096f801"
format_stamp: "Formatted at 2026-09-25 02:30:47 on dist-test-slave-dn4n"
server_key: "d3c80a27ade308bb3b065ac81d9304af"
server_key_iv: "f76e0fa228cd4c684e349fe88a738916"
server_key_version: "encryptionkey@0"
I20260925 02:30:47.145525 32502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance:
uuid: "0535c7e43b6744dd88a10837c096f801"
format_stamp: "Formatted at 2026-09-25 02:30:47 on dist-test-slave-dn4n"
server_key: "d3c80a27ade308bb3b065ac81d9304af"
server_key_iv: "f76e0fa228cd4c684e349fe88a738916"
server_key_version: "encryptionkey@0"
I20260925 02:30:47.146664 32502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:30:47.147246 32517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:47.147400 32502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:47.147485 32502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/wal
uuid: "0535c7e43b6744dd88a10837c096f801"
format_stamp: "Formatted at 2026-09-25 02:30:47 on dist-test-slave-dn4n"
server_key: "d3c80a27ade308bb3b065ac81d9304af"
server_key_iv: "f76e0fa228cd4c684e349fe88a738916"
server_key_version: "encryptionkey@0"
I20260925 02:30:47.147557 32502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:47.162415 32502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:47.162657 32502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:47.162787 32502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:47.162976 32502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 02:30:47.163269 32502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 02:30:47.163318 32502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:47.163352 32502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 02:30:47.163375 32502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:47.168704 32502 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.195:43645
I20260925 02:30:47.168753 32631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.195:43645 every 8 connection(s)
I20260925 02:30:47.169006 32502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/data/info.pb
I20260925 02:30:47.173506 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.169631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.195:43281 (local address 127.11.220.254:33855)
0925 02:30:47.169697 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:47.169699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:47.169860 (+ 161us) server_negotiation.cc:407] Connection header received
0925 02:30:47.170232 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:47.170233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:47.170242 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:47.170296 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:47.170881 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.171417 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.172299 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.172522 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.172570 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.173315 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:47.173316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:47.173316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:47.173320 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:47.173330 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:47.173333 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:47.173367 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:47.173429 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 02:30:47.173451 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:30:47.173656 32634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.169580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:33855 (local address 127.11.220.195:43281)
0925 02:30:47.169749 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:47.169783 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:47.170156 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:47.170340 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:47.170343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:47.170350 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:30:47.170809 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.170813 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.171497 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.171498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.172227 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.172228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.172584 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.172585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.172700 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.173190 (+ 490us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 02:30:47.173190 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:30:47.173226 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 02:30:47.173245 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:30:47.173369 (+ 124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:30:47.173371 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:30:47.173373 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:30:47.173414 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 02:30:47.173478 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260925 02:30:47.173970 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 32502
I20260925 02:30:47.174099 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/wal/instance
I20260925 02:30:47.174190 32632 heartbeater.cc:347] Connected to a master server at 127.11.220.254:33855
I20260925 02:30:47.174278 32632 heartbeater.cc:464] Registering TS with master...
I20260925 02:30:47.174294 12147 external_mini_cluster.cc:1428] Setting key f9e2200d87c92291112c70e237b92e85
I20260925 02:30:47.174448 32632 heartbeater.cc:511] Master 127.11.220.254:33855 requested a full tablet report, sending...
I20260925 02:30:47.174799 32171 ts_manager.cc:194] Registered new tserver with Master: 0535c7e43b6744dd88a10837c096f801 (127.11.220.195:43645)
I20260925 02:30:47.175239 32171 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.220.195:43281
I20260925 02:30:47.175251 12147 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 02:30:47.178938 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.176507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.11.220.254:33855)
0925 02:30:47.176547 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:47.176549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:47.176659 (+ 110us) server_negotiation.cc:407] Connection header received
0925 02:30:47.176699 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:47.176700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:47.176708 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:47.176756 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:47.176878 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.177291 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.177990 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.178160 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.178207 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.178797 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:47.178798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:47.178798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:47.178801 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:47.178809 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:47.178811 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:47.178845 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:47.178881 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 02:30:47.178897 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:30:47.180151 32171 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 02:30:47.181910 32171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 02:30:47.187917 32642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.184072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42232 (local address 127.11.220.195:43645)
0925 02:30:47.184226 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:47.184230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:47.184558 (+ 328us) server_negotiation.cc:407] Connection header received
0925 02:30:47.184761 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:47.184763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:47.184778 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:47.184849 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:47.185045 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.185996 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.186997 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.187543 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.187634 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.187710 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:47.187803 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 02:30:47.187836 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260925 02:30:47.187917 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.183930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:43645 (local address 127.0.0.1:42232)
0925 02:30:47.184433 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:47.184494 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:47.184603 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:47.184903 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:47.184904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:47.184906 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:47.184956 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.184958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.186088 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.186089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.186937 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.186940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.187638 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.187639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.187719 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.187732 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 02:30:47.187794 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 02:30:47.187842 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":25,"thread_start_us":62,"threads_started":1}
I20260925 02:30:47.188097 32645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.184232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38574 (local address 127.11.220.194:39499)
0925 02:30:47.184404 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:47.184408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:47.184679 (+ 271us) server_negotiation.cc:407] Connection header received
0925 02:30:47.184723 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:47.184726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:47.184743 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:47.184811 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:47.185078 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.185724 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.186911 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.187577 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.187644 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.187718 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:47.187962 (+ 244us) server_negotiation.cc:299] Negotiation successful
0925 02:30:47.188022 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260925 02:30:47.188247 32643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.183930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.194:39499 (local address 127.0.0.1:38574)
0925 02:30:47.184434 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:47.184506 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:47.184590 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:47.184889 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:47.184891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:47.184893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:47.184946 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.184948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.185814 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.185815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.186834 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.186836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.187754 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.187755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.187820 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.187824 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:30:47.187873 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 02:30:47.187921 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":54,"threads_started":1}
I20260925 02:30:47.188413 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.184125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:36147 (local address 127.0.0.1:51892)
0925 02:30:47.184660 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:47.184671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:47.184748 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:47.185012 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:47.185012 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:47.185014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:47.185063 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.185064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.186171 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.186172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.187147 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.187149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.188138 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.188139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.188256 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.188259 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 02:30:47.188286 (+ 27us) client_negotiation.cc:240] Negotiation successful
0925 02:30:47.188343 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":59,"threads_started":1}
I20260925 02:30:47.188791 32644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.184220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.11.220.193:36147)
0925 02:30:47.184529 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:47.184533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:47.184852 (+ 319us) server_negotiation.cc:407] Connection header received
0925 02:30:47.184898 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:47.184900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:47.184915 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:47.185012 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:47.185137 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.186095 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.187229 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.187942 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.188538 (+ 596us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.188627 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:47.188652 (+ 25us) server_negotiation.cc:299] Negotiation successful
0925 02:30:47.188694 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":97,"threads_started":1}
I20260925 02:30:47.189042 32430 tablet_service.cc:1467] Processing CreateTablet for tablet 3c62ebde6ac247b998dcb6a618510a3f (DEFAULT_TABLE table=test-table [id=2f7c38c9cfd74c30abe615c7ef9c94c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:47.189380 32430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c62ebde6ac247b998dcb6a618510a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:47.189335 32294 tablet_service.cc:1467] Processing CreateTablet for tablet 3c62ebde6ac247b998dcb6a618510a3f (DEFAULT_TABLE table=test-table [id=2f7c38c9cfd74c30abe615c7ef9c94c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:47.189563 32294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c62ebde6ac247b998dcb6a618510a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:47.191797 32647 tablet_bootstrap.cc:492] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99: Bootstrap starting.
I20260925 02:30:47.191991 32565 tablet_service.cc:1467] Processing CreateTablet for tablet 3c62ebde6ac247b998dcb6a618510a3f (DEFAULT_TABLE table=test-table [id=2f7c38c9cfd74c30abe615c7ef9c94c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 02:30:47.192170 32646 tablet_bootstrap.cc:492] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: Bootstrap starting.
I20260925 02:30:47.192292 32565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c62ebde6ac247b998dcb6a618510a3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:47.192427 32647 tablet_bootstrap.cc:654] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:47.192757 32646 tablet_bootstrap.cc:654] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:47.192775 32647 log.cc:824] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:47.193104 32646 log.cc:824] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:47.193538 32647 tablet_bootstrap.cc:492] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99: No bootstrap required, opened a new log
I20260925 02:30:47.193611 32647 ts_tablet_manager.cc:1397] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:47.193771 32646 tablet_bootstrap.cc:492] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: No bootstrap required, opened a new log
I20260925 02:30:47.193842 32646 ts_tablet_manager.cc:1397] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:47.195284 32650 tablet_bootstrap.cc:492] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: Bootstrap starting.
I20260925 02:30:47.195731 32646 raft_consensus.cc:348] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 } } peers { permanent_uuid: "24f69131c4334135b2a242442acac749" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 39499 } } peers { permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 } }
I20260925 02:30:47.195897 32646 raft_consensus.cc:374] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:47.195914 32650 tablet_bootstrap.cc:654] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:47.195937 32646 raft_consensus.cc:729] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f69131c4334135b2a242442acac749, State: Initialized, Role: FOLLOWER
I20260925 02:30:47.195868 32647 raft_consensus.cc:348] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 } } peers { permanent_uuid: "24f69131c4334135b2a242442acac749" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 39499 } } peers { permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 } }
I20260925 02:30:47.196024 32647 raft_consensus.cc:374] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:47.196064 32647 raft_consensus.cc:729] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8a80c43759d4dc78c2a7b59a45d8c99, State: Initialized, Role: FOLLOWER
I20260925 02:30:47.196092 32646 consensus_queue.cc:260] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 } } peers { permanent_uuid: "24f69131c4334135b2a242442acac749" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 39499 } } peers { permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 } }
I20260925 02:30:47.196177 32647 consensus_queue.cc:260] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 } } peers { permanent_uuid: "24f69131c4334135b2a242442acac749" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 39499 } } peers { permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 } }
I20260925 02:30:47.196193 32650 log.cc:824] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:47.196354 32646 ts_tablet_manager.cc:1428] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 02:30:47.196420 32647 ts_tablet_manager.cc:1428] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 02:30:47.196478 32499 heartbeater.cc:503] Master 127.11.220.254:33855 was elected leader, sending a full tablet report...
I20260925 02:30:47.196437 32363 heartbeater.cc:503] Master 127.11.220.254:33855 was elected leader, sending a full tablet report...
I20260925 02:30:47.197278 32650 tablet_bootstrap.cc:492] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: No bootstrap required, opened a new log
I20260925 02:30:47.197345 32650 ts_tablet_manager.cc:1397] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 02:30:47.199023 32650 raft_consensus.cc:348] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 } } peers { permanent_uuid: "24f69131c4334135b2a242442acac749" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 39499 } } peers { permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 } }
I20260925 02:30:47.199121 32650 raft_consensus.cc:374] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:47.199139 32650 raft_consensus.cc:729] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0535c7e43b6744dd88a10837c096f801, State: Initialized, Role: FOLLOWER
I20260925 02:30:47.199250 32650 consensus_queue.cc:260] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 } } peers { permanent_uuid: "24f69131c4334135b2a242442acac749" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 39499 } } peers { permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 } }
I20260925 02:30:47.199483 32650 ts_tablet_manager.cc:1428] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 02:30:47.199524 32632 heartbeater.cc:503] Master 127.11.220.254:33855 was elected leader, sending a full tablet report...
I20260925 02:30:47.267606 32651 raft_consensus.cc:482] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 02:30:47.267699 32651 raft_consensus.cc:504] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 } } peers { permanent_uuid: "24f69131c4334135b2a242442acac749" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 39499 } } peers { permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 } }
I20260925 02:30:47.268268 32651 leader_election.cc:288] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0535c7e43b6744dd88a10837c096f801 (127.11.220.195:43645), f8a80c43759d4dc78c2a7b59a45d8c99 (127.11.220.193:36147)
I20260925 02:30:47.270810 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.268419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.195:43645 (local address 127.11.220.194:56793)
0925 02:30:47.268486 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:47.268498 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:47.268539 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:47.268704 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:47.268705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:47.268707 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:47.268775 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.268777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.269376 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.269377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.270046 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.270048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.270544 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.270545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.270614 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.270616 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 02:30:47.270695 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 02:30:47.270736 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 02:30:47.270809 32642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.268532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:56793 (local address 127.11.220.195:43645)
0925 02:30:47.268593 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:47.268595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:47.268601 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:30:47.268624 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:47.268625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:47.268634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:47.268700 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:47.268842 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.269295 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.270137 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.270460 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.270544 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.270552 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:47.270701 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 02:30:47.270736 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 02:30:47.270964 32644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.268514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.220.194:38975 (local address 127.11.220.193:36147)
0925 02:30:47.268556 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:47.268558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:47.268693 (+ 135us) server_negotiation.cc:407] Connection header received
0925 02:30:47.268765 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:47.268766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:47.268776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:47.268843 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 02:30:47.269025 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.269442 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.270263 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.270594 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.270687 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.270694 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:47.270845 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 02:30:47.270882 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:30:47.271166 32655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.268431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:36147 (local address 127.11.220.194:38975)
0925 02:30:47.268629 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:47.268637 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:30:47.268705 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:30:47.268846 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:30:47.268847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:30:47.268848 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 02:30:47.268936 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.268937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.269526 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.269527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.270182 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:30:47.270183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.270688 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.270689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:30:47.270766 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.270770 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 02:30:47.270841 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 02:30:47.270994 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":125,"threads_started":1}
I20260925 02:30:47.271106 32585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c62ebde6ac247b998dcb6a618510a3f" candidate_uuid: "24f69131c4334135b2a242442acac749" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0535c7e43b6744dd88a10837c096f801" is_pre_election: true
I20260925 02:30:47.271304 32585 raft_consensus.cc:2454] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f69131c4334135b2a242442acac749 in term 0.
I20260925 02:30:47.271479 32316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c62ebde6ac247b998dcb6a618510a3f" candidate_uuid: "24f69131c4334135b2a242442acac749" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" is_pre_election: true
I20260925 02:30:47.271545 32384 leader_election.cc:302] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0535c7e43b6744dd88a10837c096f801, 24f69131c4334135b2a242442acac749; no voters:
I20260925 02:30:47.271600 32316 raft_consensus.cc:2454] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24f69131c4334135b2a242442acac749 in term 0.
I20260925 02:30:47.271678 32651 raft_consensus.cc:2781] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 02:30:47.271735 32651 raft_consensus.cc:482] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 02:30:47.271760 32651 raft_consensus.cc:3037] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:47.272447 32651 raft_consensus.cc:504] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 } } peers { permanent_uuid: "24f69131c4334135b2a242442acac749" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 39499 } } peers { permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 } }
I20260925 02:30:47.272614 32651 leader_election.cc:288] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [CANDIDATE]: Term 1 election: Requested vote from peers 0535c7e43b6744dd88a10837c096f801 (127.11.220.195:43645), f8a80c43759d4dc78c2a7b59a45d8c99 (127.11.220.193:36147)
I20260925 02:30:47.272770 32585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c62ebde6ac247b998dcb6a618510a3f" candidate_uuid: "24f69131c4334135b2a242442acac749" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0535c7e43b6744dd88a10837c096f801"
I20260925 02:30:47.272783 32316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c62ebde6ac247b998dcb6a618510a3f" candidate_uuid: "24f69131c4334135b2a242442acac749" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99"
I20260925 02:30:47.272847 32585 raft_consensus.cc:3037] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:47.272847 32316 raft_consensus.cc:3037] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:47.273684 32316 raft_consensus.cc:2454] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f69131c4334135b2a242442acac749 in term 1.
I20260925 02:30:47.273684 32585 raft_consensus.cc:2454] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24f69131c4334135b2a242442acac749 in term 1.
I20260925 02:30:47.273841 32385 leader_election.cc:302] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f69131c4334135b2a242442acac749, f8a80c43759d4dc78c2a7b59a45d8c99; no voters:
I20260925 02:30:47.273940 32651 raft_consensus.cc:2781] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:47.274078 32651 raft_consensus.cc:686] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 1 LEADER]: Becoming Leader. State: Replica: 24f69131c4334135b2a242442acac749, State: Running, Role: LEADER
I20260925 02:30:47.274174 32651 consensus_queue.cc:237] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 } } peers { permanent_uuid: "24f69131c4334135b2a242442acac749" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 39499 } } peers { permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 } }
I20260925 02:30:47.274955 32171 catalog_manager.cc:5495] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24f69131c4334135b2a242442acac749 (127.11.220.194). New cstate: current_term: 1 leader_uuid: "24f69131c4334135b2a242442acac749" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24f69131c4334135b2a242442acac749" member_type: VOTER last_known_addr { host: "127.11.220.194" port: 39499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 } health_report { overall_health: UNKNOWN } } }
I20260925 02:30:47.301625 32645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.299483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.11.220.194:39499)
0925 02:30:47.299536 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:47.299538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:47.299545 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:30:47.299606 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:47.299606 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:47.299620 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:47.299687 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:30:47.299789 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.300208 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.300947 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:47.301127 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:47.301190 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:47.301336 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:30:47.301423 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:30:47.301501 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:47.301517 (+ 16us) server_negotiation.cc:299] Negotiation successful
0925 02:30:47.301532 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 02:30:47.303186 32585 raft_consensus.cc:1261] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801 [term 1 FOLLOWER]: Refusing update from remote peer 24f69131c4334135b2a242442acac749: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:47.303186 32316 raft_consensus.cc:1261] T 3c62ebde6ac247b998dcb6a618510a3f P f8a80c43759d4dc78c2a7b59a45d8c99 [term 1 FOLLOWER]: Refusing update from remote peer 24f69131c4334135b2a242442acac749: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:30:47.303570 32651 consensus_queue.cc:1035] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0535c7e43b6744dd88a10837c096f801" member_type: VOTER last_known_addr { host: "127.11.220.195" port: 43645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:47.303702 32656 consensus_queue.cc:1035] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8a80c43759d4dc78c2a7b59a45d8c99" member_type: VOTER last_known_addr { host: "127.11.220.193" port: 36147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:30:47.306113 32660 mvcc.cc:205] Tried to move back new op lower bound from 7333082920151617536 to 7333082920035696640. Current Snapshot: MvccSnapshot[applied={T|T < 7333082920151617536}]
I20260925 02:30:47.306500 32661 mvcc.cc:205] Tried to move back new op lower bound from 7333082920151617536 to 7333082920035696640. Current Snapshot: MvccSnapshot[applied={T|T < 7333082920151617536}]
I20260925 02:30:47.306711 32659 mvcc.cc:205] Tried to move back new op lower bound from 7333082920151617536 to 7333082920035696640. Current Snapshot: MvccSnapshot[applied={T|T < 7333082920151617536 or (T in {7333082920151617536})}]
I20260925 02:30:47.308095 32170 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:47.308184 32170 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 02:30:47.308907 32170 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c0891296dd44306a62d58ae71ad7cb8: Sending DeleteTablet for 3 replicas of tablet 3c62ebde6ac247b998dcb6a618510a3f
I20260925 02:30:47.309167 32565 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c62ebde6ac247b998dcb6a618510a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:47 UTC) from {username='slave'} at 127.0.0.1:42232
I20260925 02:30:47.309253 32430 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c62ebde6ac247b998dcb6a618510a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:47 UTC) from {username='slave'} at 127.0.0.1:38574
I20260925 02:30:47.309373 32294 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c62ebde6ac247b998dcb6a618510a3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 02:30:47 UTC) from {username='slave'} at 127.0.0.1:51892
I20260925 02:30:47.309516 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 32231
I20260925 02:30:47.309435 32668 tablet_replica.cc:330] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: stopping tablet replica
I20260925 02:30:47.312238 32668 raft_consensus.cc:2229] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 02:30:47.312350 32668 raft_consensus.cc:2258] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:47.312522 32669 tablet_replica.cc:330] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: stopping tablet replica
I20260925 02:30:47.312618 32669 raft_consensus.cc:2229] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 1 LEADER]: Raft consensus shutting down.
I20260925 02:30:47.312719 32669 raft_consensus.cc:2258] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 02:30:47.312963 32668 ts_tablet_manager.cc:1905] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:47.313043 32669 ts_tablet_manager.cc:1905] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 02:30:47.314500 32669 ts_tablet_manager.cc:1918] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:47.314570 32669 log.cc:1196] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-1/wal/wals/3c62ebde6ac247b998dcb6a618510a3f
I20260925 02:30:47.314764 32669 ts_tablet_manager.cc:1939] T 3c62ebde6ac247b998dcb6a618510a3f P 24f69131c4334135b2a242442acac749: Deleting consensus metadata
I20260925 02:30:47.314775 32668 ts_tablet_manager.cc:1918] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 02:30:47.314941 32668 log.cc:1196] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: Deleting WAL directory at /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790303222243611-12147-0/minicluster-data/ts-2/wal/wals/3c62ebde6ac247b998dcb6a618510a3f
I20260925 02:30:47.315078 32158 catalog_manager.cc:4841] TS 24f69131c4334135b2a242442acac749 (127.11.220.194:39499): tablet 3c62ebde6ac247b998dcb6a618510a3f (table test-table [id=2f7c38c9cfd74c30abe615c7ef9c94c3]) successfully deleted
I20260925 02:30:47.315292 32668 ts_tablet_manager.cc:1939] T 3c62ebde6ac247b998dcb6a618510a3f P 0535c7e43b6744dd88a10837c096f801: Deleting consensus metadata
I20260925 02:30:47.315675 32157 catalog_manager.cc:4841] TS 0535c7e43b6744dd88a10837c096f801 (127.11.220.195:43645): tablet 3c62ebde6ac247b998dcb6a618510a3f (table test-table [id=2f7c38c9cfd74c30abe615c7ef9c94c3]) successfully deleted
W20260925 02:30:47.323464 32158 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.220.193:36147 (error 108)
I20260925 02:30:47.323662 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 32367
I20260925 02:30:47.323910 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:47.323769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.193:36147 (local address 127.0.0.1:51900)
0925 02:30:47.323839 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 02:30:47.323880 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.193:36147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 02:30:47.325343 32158 catalog_manager.cc:4584] TS f8a80c43759d4dc78c2a7b59a45d8c99 (127.11.220.193:36147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c62ebde6ac247b998dcb6a618510a3f: Network error: Client connection negotiation failed: client connection to 127.11.220.193:36147: connect: Connection refused (error 111)
I20260925 02:30:47.328896 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 32502
I20260925 02:30:47.333348 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 32139
2026-09-25T02:30:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (663 ms)
[----------] 6 tests from EncryptionPolicyTest (7816 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:47 dist-test-slave-dn4n krb5kdc[32675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:47 dist-test-slave-dn4n krb5kdc[32675](info): set up 2 sockets
Sep 25 02:30:47 dist-test-slave-dn4n krb5kdc[32675](info): commencing operation
krb5kdc: starting...
W20260925 02:30:48.281327 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:48 dist-test-slave-dn4n krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:30:48.383101 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:48.454990 32688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:48.455144 32688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:48.455163 32688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:48.456099 32688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:48.456128 32688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:48.456140 32688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:48.456151 32688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:48.456162 32688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:48.457355 32688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:48.457526 32688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:48.457674 32688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:48.461769 32695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:48.461876 32688 server_base.cc:1034] running on GCE node
W20260925 02:30:48.461793 32697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:48.461784 32693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:48.462199 32688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:48.462433 32688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:48.463569 32688 hybrid_clock.cc:648] HybridClock initialized: now 1790303448463559 us; error 27 us; skew 500 ppm
Sep 25 02:30:48 dist-test-slave-dn4n krb5kdc[32675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303448, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:48.465718 32688 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:30:48.466099 32688 webserver.cc:460] Webserver started at http://127.0.0.1:37013/ using document root <none> and password file <none>
I20260925 02:30:48.466324 32688 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:48.466362 32688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:48.466522 32688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:48.467459 32688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "c256da6839f549d7b2c8f11787911bba"
format_stamp: "Formatted at 2026-09-25 02:30:48 on dist-test-slave-dn4n"
server_key: "e6dc2339a770d1b292ada765052e80bb"
server_key_iv: "d36e6b62474c446921a1d876118b97a7"
server_key_version: "encryptionkey@0"
I20260925 02:30:48.467784 32688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "c256da6839f549d7b2c8f11787911bba"
format_stamp: "Formatted at 2026-09-25 02:30:48 on dist-test-slave-dn4n"
server_key: "e6dc2339a770d1b292ada765052e80bb"
server_key_iv: "d36e6b62474c446921a1d876118b97a7"
server_key_version: "encryptionkey@0"
I20260925 02:30:48.469051 32688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:30:48.469726 32704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:48.469893 32688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:48.469956 32688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "c256da6839f549d7b2c8f11787911bba"
format_stamp: "Formatted at 2026-09-25 02:30:48 on dist-test-slave-dn4n"
server_key: "e6dc2339a770d1b292ada765052e80bb"
server_key_iv: "d36e6b62474c446921a1d876118b97a7"
server_key_version: "encryptionkey@0"
I20260925 02:30:48.470031 32688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:48.479882 32688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:48.481549 32688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:48.481679 32688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:48.485128 32688 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36635
I20260925 02:30:48.485158 32757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36635 every 8 connection(s)
I20260925 02:30:48.485437 32688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:48.486198 32758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:48.487012 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 32688
I20260925 02:30:48.487118 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:48.487317 12147 external_mini_cluster.cc:1428] Setting key ccf609138d5afb98b8878d4f2f04aa91
I20260925 02:30:48.488659 32758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba: Bootstrap starting.
I20260925 02:30:48.489184 32758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:48.489460 32758 log.cc:824] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:48.490087 32758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba: No bootstrap required, opened a new log
I20260925 02:30:48.491755 32758 raft_consensus.cc:348] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256da6839f549d7b2c8f11787911bba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } }
I20260925 02:30:48.491922 32758 raft_consensus.cc:374] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:48.491974 32758 raft_consensus.cc:729] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c256da6839f549d7b2c8f11787911bba, State: Initialized, Role: FOLLOWER
I20260925 02:30:48.492101 32758 consensus_queue.cc:260] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256da6839f549d7b2c8f11787911bba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } }
I20260925 02:30:48.492182 32758 raft_consensus.cc:388] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:48.492223 32758 raft_consensus.cc:482] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:48.492269 32758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:48.492843 32758 raft_consensus.cc:504] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256da6839f549d7b2c8f11787911bba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } }
I20260925 02:30:48.492954 32758 leader_election.cc:302] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c256da6839f549d7b2c8f11787911bba; no voters:
I20260925 02:30:48.493100 32758 leader_election.cc:288] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:48.493170 32763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:48.493335 32758 sys_catalog.cc:564] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [sys.catalog]: configured and running, proceeding with master startup.
Sep 25 02:30:48 dist-test-slave-dn4n krb5kdc[32675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 02:30:48.493367 32763 raft_consensus.cc:686] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [term 1 LEADER]: Becoming Leader. State: Replica: c256da6839f549d7b2c8f11787911bba, State: Running, Role: LEADER
I20260925 02:30:48.493590 32763 consensus_queue.cc:237] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256da6839f549d7b2c8f11787911bba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } }
I20260925 02:30:48.493916 32764 sys_catalog.cc:455] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c256da6839f549d7b2c8f11787911bba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256da6839f549d7b2c8f11787911bba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } } }
I20260925 02:30:48.493990 32764 sys_catalog.cc:458] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:48.494158 32765 sys_catalog.cc:455] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [sys.catalog]: SysCatalogTable state changed. Reason: New leader c256da6839f549d7b2c8f11787911bba. Latest consensus state: current_term: 1 leader_uuid: "c256da6839f549d7b2c8f11787911bba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c256da6839f549d7b2c8f11787911bba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36635 } } }
I20260925 02:30:48.494208 32765 sys_catalog.cc:458] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:48.494601 304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:48.495199 304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:48.497412 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:48.487821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50763 (local address 127.0.0.1:36635)
0925 02:30:48.487995 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:48.488014 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:48.488044 (+ 30us) server_negotiation.cc:407] Connection header received
0925 02:30:48.488438 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:48.488462 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:48.488536 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:48.488675 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:48.489285 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:48.490178 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:48.490850 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:48.491235 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:48.491356 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:48.494040 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:48.494044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:48.494045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:48.494058 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:48.495844 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:48.496261 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:48.496264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:48.496264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:48.496361 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:48.496606 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:48.496607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:48.496607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:48.496814 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:48.496979 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:48.497180 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 02:30:48.497275 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260925 02:30:48.498342 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:30:48.503854 304 catalog_manager.cc:1347] Generated new cluster ID: 7917821a84394bedbf00277e802e5b54
I20260925 02:30:48.503902 304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:48.505486 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:48.499576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46093 (local address 127.0.0.1:36635)
0925 02:30:48.499607 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:48.499608 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:48.499724 (+ 116us) server_negotiation.cc:407] Connection header received
0925 02:30:48.499792 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:48.499793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:48.499802 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:48.499852 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:48.500342 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:48.501057 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:48.501695 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:48.501903 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:48.502014 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:48.503110 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:48.503111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:48.503111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:48.503118 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:48.504360 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:48.504752 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:48.504753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:48.504753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:48.504806 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:48.505020 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:48.505021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:48.505021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:48.505124 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:48.505224 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:48.505346 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 02:30:48.505402 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 02:30:48.515230 304 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:48.515650 304 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:48.522254 304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c256da6839f549d7b2c8f11787911bba: Generated new TSK 0
I20260925 02:30:48.522415 304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:48.526501 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 32688
W20260925 02:30:48.531142 32673 connection.cc:535] client connection to 127.0.0.1:36635 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36635): Connection reset by peer (error 104)
2026-09-25T02:30:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1192 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:48 dist-test-slave-dn4n krb5kdc[322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:48 dist-test-slave-dn4n krb5kdc[322](info): set up 2 sockets
Sep 25 02:30:48 dist-test-slave-dn4n krb5kdc[322](info): commencing operation
krb5kdc: starting...
W20260925 02:30:49.469363 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:49 dist-test-slave-dn4n krb5kdc[322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:30:49.569267 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42275
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:49.640246 335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:49.640434 335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:49.640480 335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:49.641472 335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:49.641525 335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:49.641546 335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:49.641575 335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:49.641597 335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:49.642937 335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42275
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:49.643188 335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:49.643384 335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:49.647524 342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:49.647631 345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:49.647541 343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:49.647640 335 server_base.cc:1034] running on GCE node
I20260925 02:30:49.647953 335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:49.648226 335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:49.649386 335 hybrid_clock.cc:648] HybridClock initialized: now 1790303449649373 us; error 27 us; skew 500 ppm
Sep 25 02:30:49 dist-test-slave-dn4n krb5kdc[322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303449, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:49.656978 335 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:30:49.657361 335 webserver.cc:460] Webserver started at http://127.0.0.1:38609/ using document root <none> and password file <none>
I20260925 02:30:49.657568 335 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:49.657632 335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:49.657773 335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:49.658658 335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "9d13e58b965a4a62ac6bc0ba67476684"
format_stamp: "Formatted at 2026-09-25 02:30:49 on dist-test-slave-dn4n"
server_key: "0ba8540c6eed084a5cd3dc2818407fe8"
server_key_iv: "81af2da4b8fd71066106bd3adf470629"
server_key_version: "encryptionkey@0"
I20260925 02:30:49.658970 335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "9d13e58b965a4a62ac6bc0ba67476684"
format_stamp: "Formatted at 2026-09-25 02:30:49 on dist-test-slave-dn4n"
server_key: "0ba8540c6eed084a5cd3dc2818407fe8"
server_key_iv: "81af2da4b8fd71066106bd3adf470629"
server_key_version: "encryptionkey@0"
I20260925 02:30:49.660156 335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 02:30:49.660854 352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:49.660995 335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:49.661096 335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "9d13e58b965a4a62ac6bc0ba67476684"
format_stamp: "Formatted at 2026-09-25 02:30:49 on dist-test-slave-dn4n"
server_key: "0ba8540c6eed084a5cd3dc2818407fe8"
server_key_iv: "81af2da4b8fd71066106bd3adf470629"
server_key_version: "encryptionkey@0"
I20260925 02:30:49.661161 335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:49.683657 335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:49.685221 335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:49.685357 335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:49.689109 335 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42275
I20260925 02:30:49.689136 404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42275 every 8 connection(s)
I20260925 02:30:49.689430 335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:49.690129 405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:49.691975 405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684: Bootstrap starting.
I20260925 02:30:49.692562 405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:49.692826 405 log.cc:824] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:49.693101 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 335
I20260925 02:30:49.693216 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:49.693391 12147 external_mini_cluster.cc:1428] Setting key 21827e2644c7226076f9f602326a55c2
I20260925 02:30:49.693557 405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684: No bootstrap required, opened a new log
I20260925 02:30:49.695194 405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d13e58b965a4a62ac6bc0ba67476684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42275 } }
I20260925 02:30:49.695406 405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:49.695488 405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d13e58b965a4a62ac6bc0ba67476684, State: Initialized, Role: FOLLOWER
I20260925 02:30:49.695647 405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d13e58b965a4a62ac6bc0ba67476684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42275 } }
I20260925 02:30:49.695730 405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:49.695796 405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:49.695847 405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:49.696533 405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d13e58b965a4a62ac6bc0ba67476684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42275 } }
I20260925 02:30:49.696627 405 leader_election.cc:302] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d13e58b965a4a62ac6bc0ba67476684; no voters:
I20260925 02:30:49.696753 405 leader_election.cc:288] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:49.696913 411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:49.697126 411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [term 1 LEADER]: Becoming Leader. State: Replica: 9d13e58b965a4a62ac6bc0ba67476684, State: Running, Role: LEADER
I20260925 02:30:49.696970 405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:49.697983 411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d13e58b965a4a62ac6bc0ba67476684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42275 } }
I20260925 02:30:49.698385 411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d13e58b965a4a62ac6bc0ba67476684. Latest consensus state: current_term: 1 leader_uuid: "9d13e58b965a4a62ac6bc0ba67476684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d13e58b965a4a62ac6bc0ba67476684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42275 } } }
I20260925 02:30:49.698522 411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:49.698555 410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d13e58b965a4a62ac6bc0ba67476684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d13e58b965a4a62ac6bc0ba67476684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42275 } } }
I20260925 02:30:49.698644 410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684 [sys.catalog]: This master's current role is: LEADER
W20260925 02:30:49.699143 425 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:30:49.699241 425 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:30:49.699460 426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:49.699626 426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:49.700632 409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:49.693843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36225 (local address 127.0.0.1:42275)
0925 02:30:49.694008 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:49.694016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:49.694043 (+ 27us) server_negotiation.cc:407] Connection header received
0925 02:30:49.695167 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:49.695175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:49.695248 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:49.695379 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:49.695665 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:49.697212 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:49.698223 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:49.699091 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:49.699214 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:49.699907 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:49.699911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:49.699911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:49.700117 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:49.700144 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:49.700153 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:49.700245 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:49.700332 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 02:30:49.700358 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260925 02:30:49.701363 426 catalog_manager.cc:1347] Generated new cluster ID: 612f1db01de244bebf5a310747a4879e
I20260925 02:30:49.701408 426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:49.701596 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:30:49.706121 409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:49.703085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57613 (local address 127.0.0.1:42275)
0925 02:30:49.703117 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:49.703118 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:49.703126 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:30:49.703191 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:49.703191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:49.703199 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:49.703248 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:49.703372 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:49.704009 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:49.705007 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:49.705273 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:49.705328 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:49.705896 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:49.705897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:49.705897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:49.705901 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:49.705911 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:49.705915 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:49.705962 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:49.706052 (+ 90us) server_negotiation.cc:299] Negotiation successful
0925 02:30:49.706062 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:30:49.713187 426 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:49.713662 426 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:49.718206 426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d13e58b965a4a62ac6bc0ba67476684: Generated new TSK 0
I20260925 02:30:49.718356 426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:49.724484 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 335
W20260925 02:30:49.729020 320 connection.cc:535] client connection to 127.0.0.1:42275 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42275): Connection reset by peer (error 104)
2026-09-25T02:30:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1197 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:49 dist-test-slave-dn4n krb5kdc[437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:49 dist-test-slave-dn4n krb5kdc[437](info): set up 2 sockets
Sep 25 02:30:49 dist-test-slave-dn4n krb5kdc[437](info): commencing operation
krb5kdc: starting...
W20260925 02:30:50.696923 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:50 dist-test-slave-dn4n krb5kdc[437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:30:50.801043 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34347
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:50.875581 450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:50.875751 450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:50.875771 450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:50.876868 450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:50.876904 450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:50.876919 450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:50.876931 450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:50.876968 450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:50.878350 450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34347
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:50.878530 450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:50.878746 450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:50.883414 458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:50.883440 456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:50.883441 455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:50.883591 450 server_base.cc:1034] running on GCE node
I20260925 02:30:50.883852 450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:50.884171 450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:50.885340 450 hybrid_clock.cc:648] HybridClock initialized: now 1790303450885287 us; error 44 us; skew 500 ppm
Sep 25 02:30:50 dist-test-slave-dn4n krb5kdc[437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303450, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:50.887610 450 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:30:50.888062 450 webserver.cc:460] Webserver started at http://127.0.0.1:35159/ using document root <none> and password file <none>
I20260925 02:30:50.888273 450 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:50.888334 450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:50.888499 450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:50.889439 450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "4467ce8b25db4e8083b884401e730c13"
format_stamp: "Formatted at 2026-09-25 02:30:50 on dist-test-slave-dn4n"
server_key: "5f8dd050315cbdbca45b4592c3a0aa8f"
server_key_iv: "082d4785c127ff8d053c0724d49619a3"
server_key_version: "encryptionkey@0"
I20260925 02:30:50.889799 450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "4467ce8b25db4e8083b884401e730c13"
format_stamp: "Formatted at 2026-09-25 02:30:50 on dist-test-slave-dn4n"
server_key: "5f8dd050315cbdbca45b4592c3a0aa8f"
server_key_iv: "082d4785c127ff8d053c0724d49619a3"
server_key_version: "encryptionkey@0"
I20260925 02:30:50.891052 450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 02:30:50.891721 465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:50.891885 450 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:50.891980 450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "4467ce8b25db4e8083b884401e730c13"
format_stamp: "Formatted at 2026-09-25 02:30:50 on dist-test-slave-dn4n"
server_key: "5f8dd050315cbdbca45b4592c3a0aa8f"
server_key_iv: "082d4785c127ff8d053c0724d49619a3"
server_key_version: "encryptionkey@0"
I20260925 02:30:50.892064 450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:50.895257 450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:50.896924 450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:50.897085 450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:50.900811 450 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34347
I20260925 02:30:50.900837 518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34347 every 8 connection(s)
I20260925 02:30:50.901129 450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:50.901881 519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:50.903939 519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13: Bootstrap starting.
I20260925 02:30:50.904503 519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:50.904695 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 450
I20260925 02:30:50.904803 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:50.904853 519 log.cc:824] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:50.905005 12147 external_mini_cluster.cc:1428] Setting key 75a7fa7a1b7697968e716fb8e98a80a5
I20260925 02:30:50.905613 519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13: No bootstrap required, opened a new log
I20260925 02:30:50.907294 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:50.905442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41345 (local address 127.0.0.1:34347)
0925 02:30:50.905615 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:50.905622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:50.905652 (+ 30us) server_negotiation.cc:407] Connection header received
0925 02:30:50.906100 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:50.906108 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:50.906139 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:50.906273 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:50.906877 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:50.906881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:50.906881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:50.906886 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:50.906925 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:50.906938 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:50.907041 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:50.907070 (+ 29us) server_negotiation.cc:299] Negotiation successful
0925 02:30:50.907107 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260925 02:30:50.908161 519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4467ce8b25db4e8083b884401e730c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34347 } }
I20260925 02:30:50.908265 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:30:50.908303 519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:50.908332 519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4467ce8b25db4e8083b884401e730c13, State: Initialized, Role: FOLLOWER
I20260925 02:30:50.908428 519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4467ce8b25db4e8083b884401e730c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34347 } }
I20260925 02:30:50.908536 519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:50.908587 519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:50.908629 519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:50.909286 519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4467ce8b25db4e8083b884401e730c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34347 } }
I20260925 02:30:50.909403 519 leader_election.cc:302] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4467ce8b25db4e8083b884401e730c13; no voters:
I20260925 02:30:50.909569 519 leader_election.cc:288] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:50.909601 528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:50.909749 528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [term 1 LEADER]: Becoming Leader. State: Replica: 4467ce8b25db4e8083b884401e730c13, State: Running, Role: LEADER
I20260925 02:30:50.909806 519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:50.909848 528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4467ce8b25db4e8083b884401e730c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34347 } }
I20260925 02:30:50.910137 528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4467ce8b25db4e8083b884401e730c13. Latest consensus state: current_term: 1 leader_uuid: "4467ce8b25db4e8083b884401e730c13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4467ce8b25db4e8083b884401e730c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34347 } } }
I20260925 02:30:50.910205 528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:50.910168 530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4467ce8b25db4e8083b884401e730c13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4467ce8b25db4e8083b884401e730c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34347 } } }
I20260925 02:30:50.910377 530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:50.910446 537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:50.911238 537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:50.911399 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:50.909928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52853 (local address 127.0.0.1:34347)
0925 02:30:50.909959 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:50.909961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:50.910118 (+ 157us) server_negotiation.cc:407] Connection header received
0925 02:30:50.910514 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:50.910515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:50.910519 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:50.910578 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:50.911139 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:50.911140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:50.911140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:50.911144 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:50.911156 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:50.911160 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:50.911198 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:50.911313 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 02:30:50.911326 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:30:50.912720 537 catalog_manager.cc:1347] Generated new cluster ID: f1377e5a26a645c48ed86bf700905931
I20260925 02:30:50.912791 537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:50.943614 537 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:50.943771 537 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:50.950603 537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4467ce8b25db4e8083b884401e730c13: Generated new TSK 0
I20260925 02:30:50.950774 537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:50.952294 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 450
2026-09-25T02:30:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1229 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:51 dist-test-slave-dn4n krb5kdc[551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:51 dist-test-slave-dn4n krb5kdc[551](info): set up 2 sockets
Sep 25 02:30:51 dist-test-slave-dn4n krb5kdc[551](info): commencing operation
krb5kdc: starting...
W20260925 02:30:51.912961 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:51 dist-test-slave-dn4n krb5kdc[551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:30:52.019483 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:52.092204 565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:52.092383 565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:52.092406 565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:52.093560 565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:52.093601 565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:52.093614 565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:52.093627 565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:52.093680 565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:52.094990 565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:52.095221 565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:52.095428 565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:52.099852 571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:52.099850 570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:52.099972 565 server_base.cc:1034] running on GCE node
W20260925 02:30:52.099846 573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:52.100247 565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:52.100589 565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:52.101749 565 hybrid_clock.cc:648] HybridClock initialized: now 1790303452101742 us; error 33 us; skew 500 ppm
Sep 25 02:30:52 dist-test-slave-dn4n krb5kdc[551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303452, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:52.104049 565 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:30:52.104581 565 webserver.cc:460] Webserver started at http://127.0.0.1:40883/ using document root <none> and password file <none>
I20260925 02:30:52.104825 565 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:52.104887 565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:52.105013 565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:52.105927 565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "5cde77cfe18749d38844cab3ec364295"
format_stamp: "Formatted at 2026-09-25 02:30:52 on dist-test-slave-dn4n"
server_key: "78adb22c44f262a95a263d0dbade844e"
server_key_iv: "91ba04c7bd7143ec0c57d88fcf784ea7"
server_key_version: "encryptionkey@0"
I20260925 02:30:52.106261 565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "5cde77cfe18749d38844cab3ec364295"
format_stamp: "Formatted at 2026-09-25 02:30:52 on dist-test-slave-dn4n"
server_key: "78adb22c44f262a95a263d0dbade844e"
server_key_iv: "91ba04c7bd7143ec0c57d88fcf784ea7"
server_key_version: "encryptionkey@0"
I20260925 02:30:52.107491 565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:30:52.108156 581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:52.108323 565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:52.108418 565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "5cde77cfe18749d38844cab3ec364295"
format_stamp: "Formatted at 2026-09-25 02:30:52 on dist-test-slave-dn4n"
server_key: "78adb22c44f262a95a263d0dbade844e"
server_key_iv: "91ba04c7bd7143ec0c57d88fcf784ea7"
server_key_version: "encryptionkey@0"
I20260925 02:30:52.108500 565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:52.127650 565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:52.129240 565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:52.129333 565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:52.132745 565 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33245
I20260925 02:30:52.132804 634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33245 every 8 connection(s)
I20260925 02:30:52.133128 565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:52.133520 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 565
I20260925 02:30:52.133647 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:52.133832 12147 external_mini_cluster.cc:1428] Setting key 528798066ed84883700c172790f4ae64
I20260925 02:30:52.134123 635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:52.136744 635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295: Bootstrap starting.
I20260925 02:30:52.137284 635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:52.137589 635 log.cc:824] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:52.138190 635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295: No bootstrap required, opened a new log
Sep 25 02:30:52 dist-test-slave-dn4n krb5kdc[551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 02:30:52.140093 635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde77cfe18749d38844cab3ec364295" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } }
I20260925 02:30:52.140244 635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:52.140295 635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cde77cfe18749d38844cab3ec364295, State: Initialized, Role: FOLLOWER
I20260925 02:30:52.140424 635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde77cfe18749d38844cab3ec364295" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } }
I20260925 02:30:52.140537 635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:52.140587 635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:52.140640 635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:52.141088 635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde77cfe18749d38844cab3ec364295" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } }
I20260925 02:30:52.141161 635 leader_election.cc:302] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cde77cfe18749d38844cab3ec364295; no voters:
I20260925 02:30:52.141289 635 leader_election.cc:288] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:52.141321 640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:52.141426 640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [term 1 LEADER]: Becoming Leader. State: Replica: 5cde77cfe18749d38844cab3ec364295, State: Running, Role: LEADER
I20260925 02:30:52.141526 635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:52.141517 640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde77cfe18749d38844cab3ec364295" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } }
I20260925 02:30:52.141957 641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cde77cfe18749d38844cab3ec364295. Latest consensus state: current_term: 1 leader_uuid: "5cde77cfe18749d38844cab3ec364295" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde77cfe18749d38844cab3ec364295" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } } }
I20260925 02:30:52.142056 641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:52.142318 650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:52.142699 642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cde77cfe18749d38844cab3ec364295" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cde77cfe18749d38844cab3ec364295" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33245 } } }
I20260925 02:30:52.142822 650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:52.142831 642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:52.143963 658 mvcc.cc:205] Tried to move back new op lower bound from 7333082939980324864 to 7333082939972579328. Current Snapshot: MvccSnapshot[applied={T|T < 7333082939980324864}]
I20260925 02:30:52.144105 638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:52.134270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38067 (local address 127.0.0.1:33245)
0925 02:30:52.134451 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:52.134460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:52.134488 (+ 28us) server_negotiation.cc:407] Connection header received
0925 02:30:52.135131 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:52.135137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:52.135183 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:52.135318 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:52.135935 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:52.136884 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:52.137657 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:52.138014 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:52.138151 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:52.140639 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:52.140642 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:52.140643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:52.140655 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:52.142432 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:52.142849 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:52.142852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:52.142852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:52.142980 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:52.143209 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:52.143210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:52.143210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:52.143455 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:52.143601 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:52.143721 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 02:30:52.143841 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260925 02:30:52.144811 650 catalog_manager.cc:1347] Generated new cluster ID: 93dd59764eec48cfaa4e9e6dddad1242
I20260925 02:30:52.144863 650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:52.145043 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:30:52.149691 650 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:52.150308 650 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:52.152328 638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:52.146414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35623 (local address 127.0.0.1:33245)
0925 02:30:52.146453 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:52.146454 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:52.146526 (+ 72us) server_negotiation.cc:407] Connection header received
0925 02:30:52.146575 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:52.146576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:52.146583 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:52.146626 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:52.147186 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:52.147600 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:52.148525 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:52.148811 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:52.148873 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:52.150068 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:52.150069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:52.150070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:52.150076 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:52.151267 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:52.151597 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:52.151598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:52.151598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:52.151648 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:52.151828 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:52.151829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:52.151829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:52.151919 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:52.152023 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:52.152148 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 02:30:52.152223 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 02:30:52.155656 650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cde77cfe18749d38844cab3ec364295: Generated new TSK 0
I20260925 02:30:52.155794 650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:52.159375 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 565
2026-09-25T02:30:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1205 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:52 dist-test-slave-dn4n krb5kdc[667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:52 dist-test-slave-dn4n krb5kdc[667](info): set up 2 sockets
Sep 25 02:30:52 dist-test-slave-dn4n krb5kdc[667](info): commencing operation
krb5kdc: starting...
W20260925 02:30:53.129388 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:53 dist-test-slave-dn4n krb5kdc[667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:30:53.232524 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38667
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:53.306589 681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:53.306813 681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:53.306849 681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:53.308132 681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:53.308198 681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:53.308223 681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:53.308248 681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:53.308279 681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:53.309677 681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:53.309928 681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:53.310151 681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:53.314767 686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:53.314838 681 server_base.cc:1034] running on GCE node
W20260925 02:30:53.314790 689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:53.314762 687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:53.315219 681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:53.315527 681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:53.316679 681 hybrid_clock.cc:648] HybridClock initialized: now 1790303453316663 us; error 43 us; skew 500 ppm
Sep 25 02:30:53 dist-test-slave-dn4n krb5kdc[667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303453, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:53.318851 681 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:30:53.319237 681 webserver.cc:460] Webserver started at http://127.0.0.1:40003/ using document root <none> and password file <none>
I20260925 02:30:53.319410 681 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:53.319453 681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:53.319542 681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:53.320515 681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0"
format_stamp: "Formatted at 2026-09-25 02:30:53 on dist-test-slave-dn4n"
server_key: "9c1bb3dce4abd077d9fdddab8f51c450"
server_key_iv: "924501781ba1438f15da2783425e2726"
server_key_version: "encryptionkey@0"
I20260925 02:30:53.320823 681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0"
format_stamp: "Formatted at 2026-09-25 02:30:53 on dist-test-slave-dn4n"
server_key: "9c1bb3dce4abd077d9fdddab8f51c450"
server_key_iv: "924501781ba1438f15da2783425e2726"
server_key_version: "encryptionkey@0"
I20260925 02:30:53.321997 681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:30:53.322620 696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:53.322777 681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:53.322860 681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0"
format_stamp: "Formatted at 2026-09-25 02:30:53 on dist-test-slave-dn4n"
server_key: "9c1bb3dce4abd077d9fdddab8f51c450"
server_key_iv: "924501781ba1438f15da2783425e2726"
server_key_version: "encryptionkey@0"
I20260925 02:30:53.322922 681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:53.348706 681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:53.350811 681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:53.350908 681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:53.354633 681 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38667
I20260925 02:30:53.354682 751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38667 every 8 connection(s)
I20260925 02:30:53.354990 681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:53.355798 752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:53.356617 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 681
I20260925 02:30:53.356727 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:53.356896 12147 external_mini_cluster.cc:1428] Setting key b63199f6ce81fa5df3d7f781a57bee7a
I20260925 02:30:53.358256 752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0: Bootstrap starting.
I20260925 02:30:53.358847 752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:53.359126 752 log.cc:824] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:53.359823 752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0: No bootstrap required, opened a new log
I20260925 02:30:53.362074 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:53.357353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43393 (local address 127.0.0.1:38667)
0925 02:30:53.357512 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:53.357519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:53.357540 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:30:53.358035 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:53.358040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:53.358088 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:53.358190 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:53.358453 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:53.359381 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:53.360197 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:53.360799 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:53.360905 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:53.361466 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:53.361471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:53.361471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:53.361682 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:53.361721 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:53.361736 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:53.361802 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:53.361851 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 02:30:53.361903 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260925 02:30:53.362025 752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } }
I20260925 02:30:53.362185 752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:53.362219 752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d1dbf4a1fb24ad88caa5cd593c9c5f0, State: Initialized, Role: FOLLOWER
I20260925 02:30:53.362329 752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } }
I20260925 02:30:53.362421 752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:53.362450 752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:53.362484 752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:53.363027 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:30:53.363157 752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } }
I20260925 02:30:53.363252 752 leader_election.cc:302] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d1dbf4a1fb24ad88caa5cd593c9c5f0; no voters:
I20260925 02:30:53.363394 752 leader_election.cc:288] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:53.363440 757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:53.363615 752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:53.363624 757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [term 1 LEADER]: Becoming Leader. State: Replica: 4d1dbf4a1fb24ad88caa5cd593c9c5f0, State: Running, Role: LEADER
I20260925 02:30:53.363762 757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } }
I20260925 02:30:53.364142 758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } } }
I20260925 02:30:53.364279 759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d1dbf4a1fb24ad88caa5cd593c9c5f0. Latest consensus state: current_term: 1 leader_uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1dbf4a1fb24ad88caa5cd593c9c5f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } } }
I20260925 02:30:53.364351 758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:53.364418 759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:53.364687 771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:53.365406 771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:53.366755 771 catalog_manager.cc:1347] Generated new cluster ID: ee3cab7d40334afaad5231c17450882e
I20260925 02:30:53.366798 771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:53.368137 755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:53.364880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46829 (local address 127.0.0.1:38667)
0925 02:30:53.364922 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:53.364923 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:53.365058 (+ 135us) server_negotiation.cc:407] Connection header received
0925 02:30:53.365142 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:53.365143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:53.365153 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:53.365203 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:53.365389 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:53.366065 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:53.367091 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:53.367349 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:53.367417 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:53.367936 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:53.367937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:53.367938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:53.367941 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:53.367954 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:53.367957 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:53.368010 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:53.368060 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 02:30:53.368079 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 02:30:53.377440 771 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:53.378039 771 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:53.387380 771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d1dbf4a1fb24ad88caa5cd593c9c5f0: Generated new TSK 0
I20260925 02:30:53.387531 771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:53.393185 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 681
2026-09-25T02:30:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1232 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:53 dist-test-slave-dn4n krb5kdc[783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:53 dist-test-slave-dn4n krb5kdc[783](info): set up 2 sockets
Sep 25 02:30:53 dist-test-slave-dn4n krb5kdc[783](info): commencing operation
krb5kdc: starting...
W20260925 02:30:54.361009 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:54 dist-test-slave-dn4n krb5kdc[783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:30:54.460886 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:54.532618 796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:54.532801 796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:54.532824 796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:54.534083 796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:54.534124 796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:54.534139 796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:54.534152 796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:54.534205 796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:54.535648 796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:54.535887 796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:54.536098 796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:54.540732 802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:54.540736 804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:54.540776 801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:54.541016 796 server_base.cc:1034] running on GCE node
I20260925 02:30:54.541249 796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:54.541512 796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:54.542645 796 hybrid_clock.cc:648] HybridClock initialized: now 1790303454542635 us; error 25 us; skew 500 ppm
Sep 25 02:30:54 dist-test-slave-dn4n krb5kdc[783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303454, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:54.544854 796 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:30:54.545305 796 webserver.cc:460] Webserver started at http://127.0.0.1:36919/ using document root <none> and password file <none>
I20260925 02:30:54.545521 796 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:54.545583 796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:54.545702 796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:54.546579 796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "fd75c8d06694440d90abebefb3e3bdb6"
format_stamp: "Formatted at 2026-09-25 02:30:54 on dist-test-slave-dn4n"
server_key: "bc68fb9f42e33ceeb92752c452d7a48e"
server_key_iv: "1459908e988520116aac908d59385fed"
server_key_version: "encryptionkey@0"
I20260925 02:30:54.546900 796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "fd75c8d06694440d90abebefb3e3bdb6"
format_stamp: "Formatted at 2026-09-25 02:30:54 on dist-test-slave-dn4n"
server_key: "bc68fb9f42e33ceeb92752c452d7a48e"
server_key_iv: "1459908e988520116aac908d59385fed"
server_key_version: "encryptionkey@0"
I20260925 02:30:54.548049 796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:30:54.548756 811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:54.548914 796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:54.549012 796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "fd75c8d06694440d90abebefb3e3bdb6"
format_stamp: "Formatted at 2026-09-25 02:30:54 on dist-test-slave-dn4n"
server_key: "bc68fb9f42e33ceeb92752c452d7a48e"
server_key_iv: "1459908e988520116aac908d59385fed"
server_key_version: "encryptionkey@0"
I20260925 02:30:54.549096 796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:54.551950 796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:54.553586 796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:54.553728 796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:54.557581 796 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34443
I20260925 02:30:54.557610 864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34443 every 8 connection(s)
I20260925 02:30:54.557894 796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:54.558640 865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:54.560608 865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6: Bootstrap starting.
I20260925 02:30:54.561121 865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:54.561412 865 log.cc:824] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:54.562009 865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6: No bootstrap required, opened a new log
I20260925 02:30:54.563490 865 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd75c8d06694440d90abebefb3e3bdb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } }
I20260925 02:30:54.563613 865 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:54.563637 865 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd75c8d06694440d90abebefb3e3bdb6, State: Initialized, Role: FOLLOWER
I20260925 02:30:54.563782 865 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd75c8d06694440d90abebefb3e3bdb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } }
I20260925 02:30:54.563838 865 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:54.563858 865 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:54.563923 865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:54.564252 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 796
I20260925 02:30:54.564366 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:54.564584 12147 external_mini_cluster.cc:1428] Setting key 9642d1b568c916c4930d78ee78fd8ea4
I20260925 02:30:54.564623 865 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd75c8d06694440d90abebefb3e3bdb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } }
I20260925 02:30:54.564728 865 leader_election.cc:302] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd75c8d06694440d90abebefb3e3bdb6; no voters:
I20260925 02:30:54.564911 865 leader_election.cc:288] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:54.564989 868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:54.565215 868 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [term 1 LEADER]: Becoming Leader. State: Replica: fd75c8d06694440d90abebefb3e3bdb6, State: Running, Role: LEADER
I20260925 02:30:54.565233 865 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:54.565356 868 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd75c8d06694440d90abebefb3e3bdb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } }
I20260925 02:30:54.565663 868 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd75c8d06694440d90abebefb3e3bdb6. Latest consensus state: current_term: 1 leader_uuid: "fd75c8d06694440d90abebefb3e3bdb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd75c8d06694440d90abebefb3e3bdb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } } }
I20260925 02:30:54.565793 868 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:54.566051 874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:54.566049 871 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd75c8d06694440d90abebefb3e3bdb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd75c8d06694440d90abebefb3e3bdb6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } } }
I20260925 02:30:54.566217 871 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:54.566612 874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:54.567022 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:54.565163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32937 (local address 127.0.0.1:34443)
0925 02:30:54.565453 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:54.565462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:54.565493 (+ 31us) server_negotiation.cc:407] Connection header received
0925 02:30:54.565609 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:54.565615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:54.565646 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:54.565786 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:54.566449 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:54.566452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:54.566453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:54.566457 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:54.566498 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:54.566512 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:54.566792 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:54.566822 (+ 30us) server_negotiation.cc:299] Negotiation successful
0925 02:30:54.566858 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":58,"threads_started":1}
I20260925 02:30:54.567998 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:30:54.569645 874 catalog_manager.cc:1347] Generated new cluster ID: f0b281365c0c49429c7b4c6999f791bc
I20260925 02:30:54.569700 874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:54.570513 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:54.569458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40109 (local address 127.0.0.1:34443)
0925 02:30:54.569518 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:54.569520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:54.569607 (+ 87us) server_negotiation.cc:407] Connection header received
0925 02:30:54.569699 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:54.569700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:54.569704 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:54.569751 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:54.570296 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:54.570297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:54.570297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:54.570301 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:54.570314 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:54.570318 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:54.570371 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:54.570438 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 02:30:54.570449 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 02:30:54.590773 874 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:54.590905 874 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:54.594726 874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd75c8d06694440d90abebefb3e3bdb6: Generated new TSK 0
I20260925 02:30:54.594864 874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:54.601783 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 796
2026-09-25T02:30:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1208 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:54 dist-test-slave-dn4n krb5kdc[897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:54 dist-test-slave-dn4n krb5kdc[897](info): set up 2 sockets
Sep 25 02:30:54 dist-test-slave-dn4n krb5kdc[897](info): commencing operation
krb5kdc: starting...
W20260925 02:30:55.573812 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:55 dist-test-slave-dn4n krb5kdc[897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:30:55.676206 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39371
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:55.754089 910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:55.754253 910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:55.754273 910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:55.755220 910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:55.755247 910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:55.755259 910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:55.755270 910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:55.755280 910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:55.756572 910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:55.756733 910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:55.756867 910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:55.761125 916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:55.761214 917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:55.761126 919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:55.761401 910 server_base.cc:1034] running on GCE node
I20260925 02:30:55.761628 910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:55.761890 910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:55.763023 910 hybrid_clock.cc:648] HybridClock initialized: now 1790303455763013 us; error 24 us; skew 500 ppm
Sep 25 02:30:55 dist-test-slave-dn4n krb5kdc[897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303455, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:55.770462 910 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:30:55.770915 910 webserver.cc:460] Webserver started at http://127.0.0.1:32831/ using document root <none> and password file <none>
I20260925 02:30:55.771143 910 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:55.771217 910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:55.771337 910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:55.772224 910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "9e3851ef72a342c9819e1bb5a5e7c24c"
format_stamp: "Formatted at 2026-09-25 02:30:55 on dist-test-slave-dn4n"
server_key: "90fd8795e425e237ea8864049a0cd44a"
server_key_iv: "33a63e244eb70ea94b74d92369e70e1e"
server_key_version: "encryptionkey@0"
I20260925 02:30:55.772575 910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "9e3851ef72a342c9819e1bb5a5e7c24c"
format_stamp: "Formatted at 2026-09-25 02:30:55 on dist-test-slave-dn4n"
server_key: "90fd8795e425e237ea8864049a0cd44a"
server_key_iv: "33a63e244eb70ea94b74d92369e70e1e"
server_key_version: "encryptionkey@0"
I20260925 02:30:55.773895 910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:30:55.774590 926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:55.774758 910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:55.774853 910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "9e3851ef72a342c9819e1bb5a5e7c24c"
format_stamp: "Formatted at 2026-09-25 02:30:55 on dist-test-slave-dn4n"
server_key: "90fd8795e425e237ea8864049a0cd44a"
server_key_iv: "33a63e244eb70ea94b74d92369e70e1e"
server_key_version: "encryptionkey@0"
I20260925 02:30:55.774935 910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:55.784020 910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:55.785559 910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:55.785714 910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:55.789450 910 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39371
I20260925 02:30:55.789480 978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39371 every 8 connection(s)
I20260925 02:30:55.789800 910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:55.789964 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 910
I20260925 02:30:55.790068 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:55.790277 12147 external_mini_cluster.cc:1428] Setting key bad7adbfce0fc81dc0a24e2eb026fe60
I20260925 02:30:55.790855 979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:55.793648 979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c: Bootstrap starting.
I20260925 02:30:55.794277 979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:55.794548 979 log.cc:824] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:55.795185 979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c: No bootstrap required, opened a new log
Sep 25 02:30:55 dist-test-slave-dn4n krb5kdc[897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 02:30:55.797061 979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3851ef72a342c9819e1bb5a5e7c24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39371 } }
I20260925 02:30:55.797240 979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:55.797339 979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e3851ef72a342c9819e1bb5a5e7c24c, State: Initialized, Role: FOLLOWER
I20260925 02:30:55.797523 979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3851ef72a342c9819e1bb5a5e7c24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39371 } }
I20260925 02:30:55.797631 979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:55.797679 979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:55.797729 979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:55.798367 979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3851ef72a342c9819e1bb5a5e7c24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39371 } }
I20260925 02:30:55.798462 979 leader_election.cc:302] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e3851ef72a342c9819e1bb5a5e7c24c; no voters:
I20260925 02:30:55.798606 979 leader_election.cc:288] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:55.798653 984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:55.798805 984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [term 1 LEADER]: Becoming Leader. State: Replica: 9e3851ef72a342c9819e1bb5a5e7c24c, State: Running, Role: LEADER
I20260925 02:30:55.798866 979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:55.798918 984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3851ef72a342c9819e1bb5a5e7c24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39371 } }
I20260925 02:30:55.799337 986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e3851ef72a342c9819e1bb5a5e7c24c. Latest consensus state: current_term: 1 leader_uuid: "9e3851ef72a342c9819e1bb5a5e7c24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3851ef72a342c9819e1bb5a5e7c24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39371 } } }
I20260925 02:30:55.799425 986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:55.799342 985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e3851ef72a342c9819e1bb5a5e7c24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3851ef72a342c9819e1bb5a5e7c24c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39371 } } }
I20260925 02:30:55.799727 985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:55.799710 992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:55.800563 992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:55.801172 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:55.790788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:39635 (local address 127.0.0.1:39371)
0925 02:30:55.790966 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:55.790974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:55.791005 (+ 31us) server_negotiation.cc:407] Connection header received
0925 02:30:55.791370 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:55.791375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:55.791422 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:55.791523 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:55.792262 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:55.793134 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:55.794148 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:55.794548 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:55.794656 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:55.796892 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:55.796897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:55.796897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:55.796908 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:55.798863 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:55.799339 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:55.799342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:55.799342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:55.799893 (+ 551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:55.800200 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:55.800202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:55.800202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:55.800529 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:55.800684 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:55.800832 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 02:30:55.800928 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260925 02:30:55.802168 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:30:55.802328 992 catalog_manager.cc:1347] Generated new cluster ID: b4f42a1b290f4312a392167a2742f39f
I20260925 02:30:55.802368 992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:55.818349 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:55.804763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:42071 (local address 127.0.0.1:39371)
0925 02:30:55.804796 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:55.804798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:55.812741 (+ 7943us) server_negotiation.cc:407] Connection header received
0925 02:30:55.812800 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:55.812801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:55.812810 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:55.812857 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:55.813456 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:55.813902 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:55.814568 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:55.814768 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:55.814844 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:55.815880 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:55.815881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:55.815882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:55.815889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:55.817297 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:55.817635 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:55.817636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:55.817636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:55.817683 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:55.817894 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:55.817895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:55.817895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:55.817992 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:55.818091 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:55.818204 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 02:30:55.818258 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:30:55.819552 992 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:55.822081 992 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:55.826356 992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e3851ef72a342c9819e1bb5a5e7c24c: Generated new TSK 0
I20260925 02:30:55.826551 992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:55.829061 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 910
W20260925 02:30:55.833295 895 connection.cc:535] client connection to 127.0.0.1:39371 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39371): Connection reset by peer (error 104)
2026-09-25T02:30:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1227 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:55 dist-test-slave-dn4n krb5kdc[1010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:55 dist-test-slave-dn4n krb5kdc[1010](info): set up 2 sockets
Sep 25 02:30:55 dist-test-slave-dn4n krb5kdc[1010](info): commencing operation
krb5kdc: starting...
W20260925 02:30:56.789289 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:56 dist-test-slave-dn4n krb5kdc[1010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:30:56.891785 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:56.964066 1024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:56.964262 1024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:56.964298 1024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:56.965364 1024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:56.965425 1024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:56.965445 1024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:56.965464 1024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:56.965497 1024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:56.966784 1024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:56.967018 1024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:56.967228 1024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:56.971652 1029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:56.971662 1032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:56.971732 1024 server_base.cc:1034] running on GCE node
W20260925 02:30:56.971664 1030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:56.972149 1024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:56.972414 1024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:56.973595 1024 hybrid_clock.cc:648] HybridClock initialized: now 1790303456973580 us; error 29 us; skew 500 ppm
Sep 25 02:30:56 dist-test-slave-dn4n krb5kdc[1010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:56.975876 1024 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:30:56.976367 1024 webserver.cc:460] Webserver started at http://127.0.0.1:38005/ using document root <none> and password file <none>
I20260925 02:30:56.976615 1024 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:56.976662 1024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:56.976747 1024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:56.977690 1024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "c5a2d9bd365549789e2c0e26cad7c850"
format_stamp: "Formatted at 2026-09-25 02:30:56 on dist-test-slave-dn4n"
server_key: "9467abc07335b0b446da8511183a2313"
server_key_iv: "e5505cab31da6f61d82cde515d279270"
server_key_version: "encryptionkey@0"
I20260925 02:30:56.977974 1024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "c5a2d9bd365549789e2c0e26cad7c850"
format_stamp: "Formatted at 2026-09-25 02:30:56 on dist-test-slave-dn4n"
server_key: "9467abc07335b0b446da8511183a2313"
server_key_iv: "e5505cab31da6f61d82cde515d279270"
server_key_version: "encryptionkey@0"
I20260925 02:30:56.979154 1024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 02:30:56.979790 1039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:56.979993 1024 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:56.980077 1024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "c5a2d9bd365549789e2c0e26cad7c850"
format_stamp: "Formatted at 2026-09-25 02:30:56 on dist-test-slave-dn4n"
server_key: "9467abc07335b0b446da8511183a2313"
server_key_iv: "e5505cab31da6f61d82cde515d279270"
server_key_version: "encryptionkey@0"
I20260925 02:30:56.980126 1024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:57.003325 1024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:57.004915 1024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:57.005016 1024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:57.008330 1024 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33391
I20260925 02:30:57.008399 1092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33391 every 8 connection(s)
I20260925 02:30:57.008687 1024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:57.009522 1093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:57.011452 1093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850: Bootstrap starting.
I20260925 02:30:57.011919 1093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:57.012126 1093 log.cc:824] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:57.012792 1093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850: No bootstrap required, opened a new log
I20260925 02:30:57.014274 1093 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a2d9bd365549789e2c0e26cad7c850" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33391 } }
I20260925 02:30:57.014390 1093 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:57.014410 1093 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a2d9bd365549789e2c0e26cad7c850, State: Initialized, Role: FOLLOWER
I20260925 02:30:57.014508 1093 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a2d9bd365549789e2c0e26cad7c850" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33391 } }
I20260925 02:30:57.014552 1093 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:57.014569 1093 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:57.014592 1093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:57.014999 1093 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a2d9bd365549789e2c0e26cad7c850" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33391 } }
I20260925 02:30:57.015070 1093 leader_election.cc:302] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5a2d9bd365549789e2c0e26cad7c850; no voters:
I20260925 02:30:57.015177 1093 leader_election.cc:288] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:57.015223 1097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:57.015394 1097 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [term 1 LEADER]: Becoming Leader. State: Replica: c5a2d9bd365549789e2c0e26cad7c850, State: Running, Role: LEADER
I20260925 02:30:57.015431 1093 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:57.015477 1097 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a2d9bd365549789e2c0e26cad7c850" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33391 } }
I20260925 02:30:57.016036 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 1024
I20260925 02:30:57.016012 1099 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5a2d9bd365549789e2c0e26cad7c850. Latest consensus state: current_term: 1 leader_uuid: "c5a2d9bd365549789e2c0e26cad7c850" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a2d9bd365549789e2c0e26cad7c850" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33391 } } }
I20260925 02:30:57.016101 1099 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:57.016127 1098 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5a2d9bd365549789e2c0e26cad7c850" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a2d9bd365549789e2c0e26cad7c850" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33391 } } }
I20260925 02:30:57.016165 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:57.016180 1098 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:57.016342 12147 external_mini_cluster.cc:1428] Setting key be4d81ea591f9a9e6cf0af3b32100939
W20260925 02:30:57.017324 1113 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:30:57.017378 1113 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:30:57.017606 1106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:57.017788 1106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:57.019250 1106 catalog_manager.cc:1347] Generated new cluster ID: e119cd2a1d804763b150534b1ba4090e
I20260925 02:30:57.019296 1106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:57.021808 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:57.016990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:52523 (local address 127.0.0.1:33391)
0925 02:30:57.017153 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:57.017162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:57.017193 (+ 31us) server_negotiation.cc:407] Connection header received
0925 02:30:57.017309 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:57.017315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:57.017368 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:57.017491 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:57.017812 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:57.018926 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:57.019948 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:57.020530 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:57.020620 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:57.021229 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:57.021233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:57.021233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:57.021407 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:57.021437 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:57.021449 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:57.021522 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:57.021619 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 02:30:57.021640 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260925 02:30:57.022634 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:30:57.027096 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:57.023968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:59423 (local address 127.0.0.1:33391)
0925 02:30:57.024006 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:57.024007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:57.024131 (+ 124us) server_negotiation.cc:407] Connection header received
0925 02:30:57.024151 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:57.024152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:57.024161 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:57.024223 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:57.024321 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:57.025004 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:57.025868 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:57.026153 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:57.026219 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:57.026787 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:57.026788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:57.026788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:57.026792 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:57.026804 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:57.026808 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:57.026864 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:57.027009 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 02:30:57.027019 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:30:57.064582 1106 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:57.065233 1106 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:57.073769 1106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5a2d9bd365549789e2c0e26cad7c850: Generated new TSK 0
I20260925 02:30:57.073938 1106 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:57.076191 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 1024
W20260925 02:30:57.080663 1008 connection.cc:535] client connection to 127.0.0.1:33391 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33391): Connection reset by peer (error 104)
2026-09-25T02:30:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1247 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:57 dist-test-slave-dn4n krb5kdc[1125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:57 dist-test-slave-dn4n krb5kdc[1125](info): set up 2 sockets
Sep 25 02:30:57 dist-test-slave-dn4n krb5kdc[1125](info): commencing operation
krb5kdc: starting...
W20260925 02:30:58.033568 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:58 dist-test-slave-dn4n krb5kdc[1125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:30:58.136703 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41289
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:58.208536 1139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:58.208698 1139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:58.208717 1139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:58.209674 1139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:58.209705 1139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:58.209718 1139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:58.209730 1139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:58.209740 1139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:58.211021 1139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:58.211189 1139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:58.211328 1139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:58.215852 1144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:58.215842 1147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:58.215890 1139 server_base.cc:1034] running on GCE node
W20260925 02:30:58.215837 1145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:58.216333 1139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:58.216624 1139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:58.217759 1139 hybrid_clock.cc:648] HybridClock initialized: now 1790303458217741 us; error 35 us; skew 500 ppm
Sep 25 02:30:58 dist-test-slave-dn4n krb5kdc[1125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303458, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:58.219933 1139 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:30:58.220335 1139 webserver.cc:460] Webserver started at http://127.0.0.1:46611/ using document root <none> and password file <none>
I20260925 02:30:58.220575 1139 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:58.220619 1139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:58.220752 1139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:58.221678 1139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "f09efc45797e4089bd167456053fa451"
format_stamp: "Formatted at 2026-09-25 02:30:58 on dist-test-slave-dn4n"
server_key: "115f2ac6d91a814d78e3d9d1a035007e"
server_key_iv: "e5d4662044b359690791554549aa623f"
server_key_version: "encryptionkey@0"
I20260925 02:30:58.222007 1139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "f09efc45797e4089bd167456053fa451"
format_stamp: "Formatted at 2026-09-25 02:30:58 on dist-test-slave-dn4n"
server_key: "115f2ac6d91a814d78e3d9d1a035007e"
server_key_iv: "e5d4662044b359690791554549aa623f"
server_key_version: "encryptionkey@0"
I20260925 02:30:58.223366 1139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:30:58.224038 1154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:58.224206 1139 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:30:58.224298 1139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "f09efc45797e4089bd167456053fa451"
format_stamp: "Formatted at 2026-09-25 02:30:58 on dist-test-slave-dn4n"
server_key: "115f2ac6d91a814d78e3d9d1a035007e"
server_key_iv: "e5d4662044b359690791554549aa623f"
server_key_version: "encryptionkey@0"
I20260925 02:30:58.224378 1139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:58.227249 1139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:58.228914 1139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:58.229060 1139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:58.232657 1139 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41289
I20260925 02:30:58.232699 1207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41289 every 8 connection(s)
I20260925 02:30:58.232997 1139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:58.233781 1208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:58.235695 1208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451: Bootstrap starting.
I20260925 02:30:58.236202 1208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:58.236522 1208 log.cc:824] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:58.237130 1208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451: No bootstrap required, opened a new log
I20260925 02:30:58.238476 1208 raft_consensus.cc:348] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09efc45797e4089bd167456053fa451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41289 } }
I20260925 02:30:58.238588 1208 raft_consensus.cc:374] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:58.238638 1208 raft_consensus.cc:729] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09efc45797e4089bd167456053fa451, State: Initialized, Role: FOLLOWER
I20260925 02:30:58.238801 1208 consensus_queue.cc:260] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09efc45797e4089bd167456053fa451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41289 } }
I20260925 02:30:58.238873 1208 raft_consensus.cc:388] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:58.238919 1208 raft_consensus.cc:482] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:58.238981 1208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:58.239465 1208 raft_consensus.cc:504] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09efc45797e4089bd167456053fa451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41289 } }
I20260925 02:30:58.239570 1208 leader_election.cc:302] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f09efc45797e4089bd167456053fa451; no voters:
I20260925 02:30:58.239738 1208 leader_election.cc:288] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:58.239764 1211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:58.240072 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 1139
I20260925 02:30:58.240180 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:58.240338 12147 external_mini_cluster.cc:1428] Setting key 3b7500ecf330ab6752c9f3fb8a1f2a54
I20260925 02:30:58.239903 1211 raft_consensus.cc:686] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [term 1 LEADER]: Becoming Leader. State: Replica: f09efc45797e4089bd167456053fa451, State: Running, Role: LEADER
I20260925 02:30:58.239980 1208 sys_catalog.cc:564] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:58.240589 1211 consensus_queue.cc:237] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09efc45797e4089bd167456053fa451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41289 } }
I20260925 02:30:58.241631 1213 sys_catalog.cc:455] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f09efc45797e4089bd167456053fa451. Latest consensus state: current_term: 1 leader_uuid: "f09efc45797e4089bd167456053fa451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09efc45797e4089bd167456053fa451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41289 } } }
I20260925 02:30:58.241789 1213 sys_catalog.cc:458] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:58.241636 1212 sys_catalog.cc:455] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f09efc45797e4089bd167456053fa451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f09efc45797e4089bd167456053fa451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41289 } } }
I20260925 02:30:58.242023 1212 sys_catalog.cc:458] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:58.242034 1223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:58.242586 1223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:58.242794 1215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:58.240703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:58969 (local address 127.0.0.1:41289)
0925 02:30:58.240848 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:58.240856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:58.240890 (+ 34us) server_negotiation.cc:407] Connection header received
0925 02:30:58.241024 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:58.241031 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:58.241063 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:58.241203 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:58.242186 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:58.242189 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:58.242190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:58.242195 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:58.242238 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:58.242255 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:58.242318 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:58.242554 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 02:30:58.242600 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
I20260925 02:30:58.243809 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:30:58.244320 1223 catalog_manager.cc:1347] Generated new cluster ID: 050f2e5b3593465a9d2fb72cf6995afa
I20260925 02:30:58.244372 1223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:58.249897 1215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:58.248800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:40017 (local address 127.0.0.1:41289)
0925 02:30:58.248835 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:58.248836 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:58.249059 (+ 223us) server_negotiation.cc:407] Connection header received
0925 02:30:58.249130 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:58.249131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:58.249136 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:58.249177 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:58.249702 (+ 525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:58.249703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:58.249703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:30:58.249708 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:58.249718 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:30:58.249722 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:58.249762 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:58.249817 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 02:30:58.249831 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:30:58.260133 1223 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:58.260300 1223 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:30:58.267117 1223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f09efc45797e4089bd167456053fa451: Generated new TSK 0
I20260925 02:30:58.267273 1223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:58.270040 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 1139
2026-09-25T02:30:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1195 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:58 dist-test-slave-dn4n krb5kdc[1240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:58 dist-test-slave-dn4n krb5kdc[1240](info): set up 2 sockets
Sep 25 02:30:58 dist-test-slave-dn4n krb5kdc[1240](info): commencing operation
krb5kdc: starting...
W20260925 02:30:59.236995 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:30:59 dist-test-slave-dn4n krb5kdc[1240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:30:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:30:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:30:59.338649 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:30:59.410993 1253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:30:59.411170 1253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:30:59.411191 1253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:30:59.412190 1253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:30:59.412221 1253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:30:59.412235 1253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:30:59.412245 1253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:30:59.412256 1253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:30:59.413535 1253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39919
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:30:59.413712 1253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:30:59.413853 1253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:30:59.418256 1261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:59.418388 1253 server_base.cc:1034] running on GCE node
W20260925 02:30:59.418256 1258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:30:59.418262 1259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:30:59.418745 1253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:30:59.418994 1253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:30:59.420147 1253 hybrid_clock.cc:648] HybridClock initialized: now 1790303459420139 us; error 36 us; skew 500 ppm
Sep 25 02:30:59 dist-test-slave-dn4n krb5kdc[1240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303459, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:30:59.422529 1253 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:30:59.422900 1253 webserver.cc:460] Webserver started at http://127.0.0.1:45019/ using document root <none> and password file <none>
I20260925 02:30:59.423116 1253 fs_manager.cc:359] Metadata directory not provided
I20260925 02:30:59.423180 1253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:30:59.423319 1253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:30:59.424226 1253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "a1c35ec7d3634e4ea6977197a77e5323"
format_stamp: "Formatted at 2026-09-25 02:30:59 on dist-test-slave-dn4n"
server_key: "7e9097ca13daafe645e33b40238726fd"
server_key_iv: "45b52e3ef9a6484c2c47c8c168c45208"
server_key_version: "encryptionkey@0"
I20260925 02:30:59.424592 1253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "a1c35ec7d3634e4ea6977197a77e5323"
format_stamp: "Formatted at 2026-09-25 02:30:59 on dist-test-slave-dn4n"
server_key: "7e9097ca13daafe645e33b40238726fd"
server_key_iv: "45b52e3ef9a6484c2c47c8c168c45208"
server_key_version: "encryptionkey@0"
I20260925 02:30:59.425761 1253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:30:59.426378 1268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:30:59.426546 1253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:30:59.426635 1253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "a1c35ec7d3634e4ea6977197a77e5323"
format_stamp: "Formatted at 2026-09-25 02:30:59 on dist-test-slave-dn4n"
server_key: "7e9097ca13daafe645e33b40238726fd"
server_key_iv: "45b52e3ef9a6484c2c47c8c168c45208"
server_key_version: "encryptionkey@0"
I20260925 02:30:59.426712 1253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:30:59.460799 1253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:30:59.462723 1253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:30:59.462882 1253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:30:59.466578 1253 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39919
I20260925 02:30:59.466609 1322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39919 every 8 connection(s)
I20260925 02:30:59.466900 1253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:30:59.467653 1323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:30:59.469542 1323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323: Bootstrap starting.
I20260925 02:30:59.470055 1323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323: Neither blocks nor log segments found. Creating new log.
I20260925 02:30:59.470286 1323 log.cc:824] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323: Log is configured to *not* fsync() on all Append() calls
I20260925 02:30:59.470880 1323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323: No bootstrap required, opened a new log
I20260925 02:30:59.472249 1323 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c35ec7d3634e4ea6977197a77e5323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260925 02:30:59.472359 1323 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:30:59.472378 1323 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1c35ec7d3634e4ea6977197a77e5323, State: Initialized, Role: FOLLOWER
I20260925 02:30:59.472520 1323 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c35ec7d3634e4ea6977197a77e5323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260925 02:30:59.472595 1323 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:30:59.472615 1323 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:30:59.472640 1323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:30:59.473132 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 1253
I20260925 02:30:59.473136 1323 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c35ec7d3634e4ea6977197a77e5323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260925 02:30:59.473239 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:30:59.473238 1323 leader_election.cc:302] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1c35ec7d3634e4ea6977197a77e5323; no voters:
I20260925 02:30:59.473384 1323 leader_election.cc:288] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:30:59.473438 12147 external_mini_cluster.cc:1428] Setting key 54babde039f085cc6fc9116a09ad0cd7
I20260925 02:30:59.473431 1326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:30:59.473572 1326 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [term 1 LEADER]: Becoming Leader. State: Replica: a1c35ec7d3634e4ea6977197a77e5323, State: Running, Role: LEADER
I20260925 02:30:59.473621 1323 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:30:59.473717 1326 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c35ec7d3634e4ea6977197a77e5323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260925 02:30:59.474290 1328 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1c35ec7d3634e4ea6977197a77e5323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c35ec7d3634e4ea6977197a77e5323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } } }
I20260925 02:30:59.474412 1328 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:59.474404 1327 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1c35ec7d3634e4ea6977197a77e5323. Latest consensus state: current_term: 1 leader_uuid: "a1c35ec7d3634e4ea6977197a77e5323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c35ec7d3634e4ea6977197a77e5323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } } }
I20260925 02:30:59.474745 1327 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323 [sys.catalog]: This master's current role is: LEADER
I20260925 02:30:59.474910 1337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:30:59.475430 1337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:30:59.476941 1337 catalog_manager.cc:1347] Generated new cluster ID: a2aa022e52c546c29bfd021e13741774
I20260925 02:30:59.476994 1337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:30:59.479863 1337 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:30:59.480334 1337 catalog_manager.cc:1504] Loading token signing keys...
Sep 25 02:30:59 dist-test-slave-dn4n krb5kdc[1240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 02:30:59.484858 1330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:59.473966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:60827 (local address 127.0.0.1:39919)
0925 02:30:59.474105 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:59.474112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:59.474536 (+ 424us) server_negotiation.cc:407] Connection header received
0925 02:30:59.474745 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:59.474753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:59.474810 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:59.474937 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:59.475890 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:59.476888 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:59.477919 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:59.478456 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:59.478543 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:59.481096 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:59.481099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:59.481099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:59.481109 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:59.483333 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:59.483735 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:59.483738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:59.483738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:59.483827 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:59.484073 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:59.484073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:59.484074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:59.484311 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:59.484436 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:59.484599 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 02:30:59.484682 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":62,"threads_started":1}
I20260925 02:30:59.485934 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:30:59.492393 1337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1c35ec7d3634e4ea6977197a77e5323: Generated new TSK 0
I20260925 02:30:59.492558 1337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:30:59.493647 1330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:30:59.487181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:52331 (local address 127.0.0.1:39919)
0925 02:30:59.487208 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0925 02:30:59.487210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:30:59.487319 (+ 109us) server_negotiation.cc:407] Connection header received
0925 02:30:59.487355 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:30:59.487356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:30:59.487363 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:30:59.487411 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:30:59.487932 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:59.488654 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:59.489602 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:30:59.489875 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:30:59.489929 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:30:59.491191 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:30:59.491192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:30:59.491192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:30:59.491198 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:30:59.492609 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:59.492994 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:59.492995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:59.492995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:59.493039 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:30:59.493264 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:30:59.493265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:30:59.493265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:30:59.493347 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:30:59.493412 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:30:59.493558 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 02:30:59.493601 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260925 02:30:59.495406 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 1253
W20260925 02:30:59.499456 1238 connection.cc:535] client connection to 127.0.0.1:39919 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39919): Connection reset by peer (error 104)
2026-09-25T02:30:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1222 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:30:59 dist-test-slave-dn4n krb5kdc[1354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:30:59 dist-test-slave-dn4n krb5kdc[1354](info): set up 2 sockets
Sep 25 02:30:59 dist-test-slave-dn4n krb5kdc[1354](info): commencing operation
krb5kdc: starting...
W20260925 02:31:00.445281 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:31:00 dist-test-slave-dn4n krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:31:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:31:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:31:00.545239 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:31:00.618046 1368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:00.618237 1368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:00.618269 1368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:00.619285 1368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:00.619345 1368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:00.619367 1368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:00.619393 1368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:00.619422 1368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:00.620741 1368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39491
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:00.620982 1368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:00.621177 1368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:00.625739 1374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:00.625767 1376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:00.625742 1373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:00.625942 1368 server_base.cc:1034] running on GCE node
I20260925 02:31:00.626119 1368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:00.626317 1368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:00.627482 1368 hybrid_clock.cc:648] HybridClock initialized: now 1790303460627476 us; error 31 us; skew 500 ppm
Sep 25 02:31:00 dist-test-slave-dn4n krb5kdc[1354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303460, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:00.629590 1368 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:31:00.629987 1368 webserver.cc:460] Webserver started at http://127.0.0.1:36223/ using document root <none> and password file <none>
I20260925 02:31:00.630183 1368 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:00.630244 1368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:00.630378 1368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:00.631261 1368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "38b70697d612452c92236d3b135afd99"
format_stamp: "Formatted at 2026-09-25 02:31:00 on dist-test-slave-dn4n"
server_key: "5c4a4a26d0708e0db021f624ca7b50d8"
server_key_iv: "87bfae1c622c60f835f4326c5ad9e134"
server_key_version: "encryptionkey@0"
I20260925 02:31:00.631570 1368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "38b70697d612452c92236d3b135afd99"
format_stamp: "Formatted at 2026-09-25 02:31:00 on dist-test-slave-dn4n"
server_key: "5c4a4a26d0708e0db021f624ca7b50d8"
server_key_iv: "87bfae1c622c60f835f4326c5ad9e134"
server_key_version: "encryptionkey@0"
I20260925 02:31:00.632714 1368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:31:00.633410 1383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:00.633584 1368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:31:00.633680 1368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "38b70697d612452c92236d3b135afd99"
format_stamp: "Formatted at 2026-09-25 02:31:00 on dist-test-slave-dn4n"
server_key: "5c4a4a26d0708e0db021f624ca7b50d8"
server_key_iv: "87bfae1c622c60f835f4326c5ad9e134"
server_key_version: "encryptionkey@0"
I20260925 02:31:00.633756 1368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:00.645573 1368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:00.647112 1368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:00.647258 1368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:00.650924 1368 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39491
I20260925 02:31:00.650950 1436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39491 every 8 connection(s)
I20260925 02:31:00.651232 1368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:31:00.651922 1437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:00.653856 1437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99: Bootstrap starting.
I20260925 02:31:00.654325 1437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:00.654552 1437 log.cc:824] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:00.655087 1437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99: No bootstrap required, opened a new log
I20260925 02:31:00.656411 1437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b70697d612452c92236d3b135afd99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260925 02:31:00.656592 1437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:31:00.656617 1437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b70697d612452c92236d3b135afd99, State: Initialized, Role: FOLLOWER
I20260925 02:31:00.656701 1437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b70697d612452c92236d3b135afd99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260925 02:31:00.656745 1437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:31:00.656762 1437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:31:00.656790 1437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:00.657186 1437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b70697d612452c92236d3b135afd99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260925 02:31:00.657258 1437 leader_election.cc:302] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38b70697d612452c92236d3b135afd99; no voters:
I20260925 02:31:00.657366 1437 leader_election.cc:288] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:31:00.657428 1440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:31:00.657585 1440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [term 1 LEADER]: Becoming Leader. State: Replica: 38b70697d612452c92236d3b135afd99, State: Running, Role: LEADER
I20260925 02:31:00.657608 1437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:00.657665 1440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b70697d612452c92236d3b135afd99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } }
I20260925 02:31:00.658035 1442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38b70697d612452c92236d3b135afd99. Latest consensus state: current_term: 1 leader_uuid: "38b70697d612452c92236d3b135afd99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b70697d612452c92236d3b135afd99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } } }
I20260925 02:31:00.658121 1442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:00.658226 1441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38b70697d612452c92236d3b135afd99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b70697d612452c92236d3b135afd99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39491 } } }
I20260925 02:31:00.658330 1441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:00.658540 1450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:31:00.658725 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 1368
I20260925 02:31:00.658830 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:31:00.659006 12147 external_mini_cluster.cc:1428] Setting key 7660600cfa5aa4279a0bdc0ee0517af2
I20260925 02:31:00.659065 1450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:31:00.660606 1450 catalog_manager.cc:1347] Generated new cluster ID: 970b575e698b4293a4b26ae1004fd378
I20260925 02:31:00.660670 1450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:31:00.664294 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:00.659452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:37097 (local address 127.0.0.1:39491)
0925 02:31:00.659632 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:00.659639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:00.659664 (+ 25us) server_negotiation.cc:407] Connection header received
0925 02:31:00.659765 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:00.659770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:00.659820 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:00.659971 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:00.660231 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:00.661420 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:00.662347 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:00.662937 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:00.663038 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:00.663673 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:00.663677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:00.663678 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:00.663874 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:00.663906 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:00.663920 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:00.663997 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:00.664075 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 02:31:00.664100 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260925 02:31:00.665135 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:31:00.669375 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:00.666419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:43165 (local address 127.0.0.1:39491)
0925 02:31:00.666446 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:00.666448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:00.666513 (+ 65us) server_negotiation.cc:407] Connection header received
0925 02:31:00.666559 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:00.666560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:00.666568 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:00.666625 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:00.666713 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:00.667371 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:00.668233 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:00.668552 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:00.668616 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:00.669163 (+ 547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:00.669165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:00.669165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:00.669169 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:00.669181 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:00.669185 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:00.669241 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:00.669304 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 02:31:00.669314 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 02:31:00.675112 1450 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:31:00.675513 1450 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:31:00.682720 1450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38b70697d612452c92236d3b135afd99: Generated new TSK 0
I20260925 02:31:00.682858 1450 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:31:00.686487 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 1368
W20260925 02:31:00.690706 1352 connection.cc:535] client connection to 127.0.0.1:39491 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39491): Connection reset by peer (error 104)
2026-09-25T02:31:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1191 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:31:00 dist-test-slave-dn4n krb5kdc[1469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:31:00 dist-test-slave-dn4n krb5kdc[1469](info): set up 2 sockets
Sep 25 02:31:00 dist-test-slave-dn4n krb5kdc[1469](info): commencing operation
krb5kdc: starting...
W20260925 02:31:01.641170 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:31:01 dist-test-slave-dn4n krb5kdc[1469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:31:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:31:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 02:31:01.743249 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38639
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:31:01.814672 1482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:01.814838 1482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:01.814859 1482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:01.815956 1482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:01.815997 1482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:01.816013 1482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:01.816066 1482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:01.816089 1482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:01.817638 1482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38639
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:01.817874 1482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:01.818100 1482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:01.822445 1488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:01.822566 1482 server_base.cc:1034] running on GCE node
W20260925 02:31:01.822455 1490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:01.822455 1487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:01.822851 1482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:01.823058 1482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:01.824223 1482 hybrid_clock.cc:648] HybridClock initialized: now 1790303461824218 us; error 31 us; skew 500 ppm
Sep 25 02:31:01 dist-test-slave-dn4n krb5kdc[1469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303461, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:01.826522 1482 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 02:31:01.826922 1482 webserver.cc:460] Webserver started at http://127.0.0.1:39003/ using document root <none> and password file <none>
I20260925 02:31:01.827128 1482 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:01.827168 1482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:01.827293 1482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:01.828176 1482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "17ea2aee4f544154aee50c7f50815173"
format_stamp: "Formatted at 2026-09-25 02:31:01 on dist-test-slave-dn4n"
server_key: "181f5ce3d7865c4dee111414fa0fbecc"
server_key_iv: "61418a770c9495cc4b445ce43801eb74"
server_key_version: "encryptionkey@0"
I20260925 02:31:01.828560 1482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "17ea2aee4f544154aee50c7f50815173"
format_stamp: "Formatted at 2026-09-25 02:31:01 on dist-test-slave-dn4n"
server_key: "181f5ce3d7865c4dee111414fa0fbecc"
server_key_iv: "61418a770c9495cc4b445ce43801eb74"
server_key_version: "encryptionkey@0"
I20260925 02:31:01.829679 1482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:31:01.830289 1497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:01.830431 1482 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:31:01.830528 1482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "17ea2aee4f544154aee50c7f50815173"
format_stamp: "Formatted at 2026-09-25 02:31:01 on dist-test-slave-dn4n"
server_key: "181f5ce3d7865c4dee111414fa0fbecc"
server_key_iv: "61418a770c9495cc4b445ce43801eb74"
server_key_version: "encryptionkey@0"
I20260925 02:31:01.830618 1482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:01.833565 1482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:01.835170 1482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:01.835315 1482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:01.838908 1482 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38639
I20260925 02:31:01.838937 1549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38639 every 8 connection(s)
I20260925 02:31:01.839203 1482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:31:01.839932 1550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:01.841998 1550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173: Bootstrap starting.
I20260925 02:31:01.842494 1550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:01.842768 1550 log.cc:824] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:01.843393 1550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173: No bootstrap required, opened a new log
I20260925 02:31:01.844856 1550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ea2aee4f544154aee50c7f50815173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } }
I20260925 02:31:01.844985 1550 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:31:01.845045 1550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17ea2aee4f544154aee50c7f50815173, State: Initialized, Role: FOLLOWER
I20260925 02:31:01.845196 1550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ea2aee4f544154aee50c7f50815173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } }
I20260925 02:31:01.845275 1550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:31:01.845320 1550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:31:01.845366 1550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:01.845806 1550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ea2aee4f544154aee50c7f50815173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } }
I20260925 02:31:01.845906 1550 leader_election.cc:302] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17ea2aee4f544154aee50c7f50815173; no voters:
I20260925 02:31:01.846096 1550 leader_election.cc:288] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:31:01.846128 1553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:31:01.846292 1553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [term 1 LEADER]: Becoming Leader. State: Replica: 17ea2aee4f544154aee50c7f50815173, State: Running, Role: LEADER
I20260925 02:31:01.846328 1550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:01.846412 1553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ea2aee4f544154aee50c7f50815173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } }
I20260925 02:31:01.846738 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 1482
I20260925 02:31:01.846765 1555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17ea2aee4f544154aee50c7f50815173. Latest consensus state: current_term: 1 leader_uuid: "17ea2aee4f544154aee50c7f50815173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ea2aee4f544154aee50c7f50815173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } } }
I20260925 02:31:01.846838 1555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:01.846853 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:31:01.846863 1554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17ea2aee4f544154aee50c7f50815173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17ea2aee4f544154aee50c7f50815173" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38639 } } }
I20260925 02:31:01.846913 1554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:01.847040 12147 external_mini_cluster.cc:1428] Setting key 323576c9fdac7667c43b3e3ed02594e6
I20260925 02:31:01.847559 1562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:31:01.848208 1562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:31:01.848985 1572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:01.847592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:57523 (local address 127.0.0.1:38639)
0925 02:31:01.847743 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:01.847752 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:01.847780 (+ 28us) server_negotiation.cc:407] Connection header received
0925 02:31:01.847873 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:01.847882 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:01.847918 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:01.848054 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:01.848625 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:01.848628 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:01.848628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:01.848632 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:01.848667 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:01.848682 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:01.848752 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:01.848812 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 02:31:01.848850 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260925 02:31:01.849779 1562 catalog_manager.cc:1347] Generated new cluster ID: 2447c75520cd4fef8a8df18879f9acc7
I20260925 02:31:01.849804 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:31:01.849826 1562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:31:01.853825 1572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:01.852747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.183:46187 (local address 127.0.0.1:38639)
0925 02:31:01.852786 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:01.852787 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:01.852920 (+ 133us) server_negotiation.cc:407] Connection header received
0925 02:31:01.852994 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:01.852995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:01.853000 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:01.853055 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:01.853603 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:01.853605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:01.853605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:01.853609 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:01.853636 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:01.853640 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:01.853689 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:01.853739 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 02:31:01.853751 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 02:31:01.856086 1562 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:31:01.856225 1562 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:31:01.859793 1562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17ea2aee4f544154aee50c7f50815173: Generated new TSK 0
I20260925 02:31:01.859935 1562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:31:01.864086 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 1482
2026-09-25T02:31:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1178 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:31:01 dist-test-slave-dn4n krb5kdc[1582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:31:01 dist-test-slave-dn4n krb5kdc[1582](info): set up 2 sockets
Sep 25 02:31:01 dist-test-slave-dn4n krb5kdc[1582](info): commencing operation
krb5kdc: starting...
W20260925 02:31:02.812930 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:31:02 dist-test-slave-dn4n krb5kdc[1582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:31:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:31:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:31:02.915370 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:34863
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:34863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:31:02.990540 1595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:02.990710 1595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:02.990728 1595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:02.991753 1595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:02.991791 1595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:02.991802 1595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:02.991814 1595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:02.991859 1595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:02.993264 1595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:34863
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:34863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:02.993494 1595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:02.993697 1595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:02.997942 1601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:02.997933 1602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:02.997961 1595 server_base.cc:1034] running on GCE node
W20260925 02:31:02.997933 1604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:02.998315 1595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:02.998597 1595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:02.999734 1595 hybrid_clock.cc:648] HybridClock initialized: now 1790303462999726 us; error 24 us; skew 500 ppm
Sep 25 02:31:02 dist-test-slave-dn4n krb5kdc[1582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303462, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:03.001981 1595 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:31:03.002389 1595 webserver.cc:460] Webserver started at http://127.11.220.254:45927/ using document root <none> and password file <none>
I20260925 02:31:03.002585 1595 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:03.002643 1595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:03.002779 1595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:03.003676 1595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "f5c487b5da4e4395b3f1a7112b26ce9f"
format_stamp: "Formatted at 2026-09-25 02:31:02 on dist-test-slave-dn4n"
server_key: "719f049a1d425c00c993857764341ec1"
server_key_iv: "764c075b3b4fcaf32ef66f7be70f3a62"
server_key_version: "encryptionkey@0"
I20260925 02:31:03.003996 1595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "f5c487b5da4e4395b3f1a7112b26ce9f"
format_stamp: "Formatted at 2026-09-25 02:31:02 on dist-test-slave-dn4n"
server_key: "719f049a1d425c00c993857764341ec1"
server_key_iv: "764c075b3b4fcaf32ef66f7be70f3a62"
server_key_version: "encryptionkey@0"
I20260925 02:31:03.005322 1595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:31:03.005925 1611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:03.006085 1595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:31:03.006175 1595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "f5c487b5da4e4395b3f1a7112b26ce9f"
format_stamp: "Formatted at 2026-09-25 02:31:02 on dist-test-slave-dn4n"
server_key: "719f049a1d425c00c993857764341ec1"
server_key_iv: "764c075b3b4fcaf32ef66f7be70f3a62"
server_key_version: "encryptionkey@0"
I20260925 02:31:03.006244 1595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:03.017917 1595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:03.019448 1595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:03.019585 1595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:03.023314 1595 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:34863
I20260925 02:31:03.023340 1663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:34863 every 8 connection(s)
I20260925 02:31:03.023658 1595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:31:03.024384 1664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:03.026368 1664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f: Bootstrap starting.
I20260925 02:31:03.026875 1664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:03.027153 1664 log.cc:824] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:03.027750 1664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f: No bootstrap required, opened a new log
I20260925 02:31:03.029244 1664 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c487b5da4e4395b3f1a7112b26ce9f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34863 } }
I20260925 02:31:03.029403 1664 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:31:03.029439 1664 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c487b5da4e4395b3f1a7112b26ce9f, State: Initialized, Role: FOLLOWER
I20260925 02:31:03.029480 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 1595
I20260925 02:31:03.029575 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:31:03.029575 1664 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c487b5da4e4395b3f1a7112b26ce9f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34863 } }
I20260925 02:31:03.029645 1664 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:31:03.029675 1664 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:31:03.029713 1664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:03.029733 12147 external_mini_cluster.cc:1428] Setting key 5bb52eb03768762ae3b9af5d4e1e34eb
I20260925 02:31:03.030550 1664 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c487b5da4e4395b3f1a7112b26ce9f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34863 } }
I20260925 02:31:03.030666 1664 leader_election.cc:302] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5c487b5da4e4395b3f1a7112b26ce9f; no voters:
I20260925 02:31:03.030822 1664 leader_election.cc:288] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:31:03.031033 1669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:31:03.031056 1664 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:03.031222 1669 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [term 1 LEADER]: Becoming Leader. State: Replica: f5c487b5da4e4395b3f1a7112b26ce9f, State: Running, Role: LEADER
I20260925 02:31:03.031378 1669 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c487b5da4e4395b3f1a7112b26ce9f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34863 } }
I20260925 02:31:03.031769 1669 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5c487b5da4e4395b3f1a7112b26ce9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c487b5da4e4395b3f1a7112b26ce9f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34863 } } }
I20260925 02:31:03.031900 1669 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:03.032058 1669 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5c487b5da4e4395b3f1a7112b26ce9f. Latest consensus state: current_term: 1 leader_uuid: "f5c487b5da4e4395b3f1a7112b26ce9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5c487b5da4e4395b3f1a7112b26ce9f" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 34863 } } }
I20260925 02:31:03.032184 1669 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:03.032346 1678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:31:03.032884 1678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:31:03.034600 1678 catalog_manager.cc:1347] Generated new cluster ID: 7404f33901f84235879073484b272b8a
I20260925 02:31:03.034657 1678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:31:03 dist-test-slave-dn4n krb5kdc[1582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:31:03.042409 1668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:03.030473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45715 (local address 127.11.220.254:34863)
0925 02:31:03.030752 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:03.030786 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:03.030877 (+ 91us) server_negotiation.cc:407] Connection header received
0925 02:31:03.031051 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:03.031059 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:03.031145 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:03.032483 (+ 1338us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:03.032659 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:03.034613 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:03.036139 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:03.036599 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:03.036711 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:03.038771 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:03.038775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:03.038775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:03.038785 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:03.040891 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:03.041247 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:03.041255 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:03.041255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:03.041334 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:03.041596 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:03.041597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:03.041597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:03.041820 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:03.041979 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:03.042084 (+ 105us) server_negotiation.cc:299] Negotiation successful
0925 02:31:03.042268 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":133,"threads_started":1}
I20260925 02:31:03.043220 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:31:03.053905 1668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:03.044539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34663 (local address 127.11.220.254:34863)
0925 02:31:03.044577 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:03.044579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:03.048601 (+ 4022us) server_negotiation.cc:407] Connection header received
0925 02:31:03.048667 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:03.048667 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:03.048676 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:03.048723 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:03.049178 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:03.049598 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:03.050242 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:03.050425 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:03.050492 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:03.051510 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:03.051511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:03.051512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:03.051518 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:03.052807 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:03.053149 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:03.053150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:03.053150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:03.053197 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:03.053404 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:03.053405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:03.053405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:03.053500 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:03.053643 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:03.053746 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 02:31:03.053805 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:31:03.056217 1678 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:31:03.056736 1678 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:31:03.062239 1678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5c487b5da4e4395b3f1a7112b26ce9f: Generated new TSK 0
I20260925 02:31:03.062418 1678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:31:03.064620 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 1595
W20260925 02:31:03.068591 1580 connection.cc:535] client connection to 127.11.220.254:34863 recv error: Network error: failed to read from TLS socket (remote: 127.11.220.254:34863): Connection reset by peer (error 104)
2026-09-25T02:31:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1200 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:31:03 dist-test-slave-dn4n krb5kdc[1695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:31:03 dist-test-slave-dn4n krb5kdc[1695](info): set up 2 sockets
Sep 25 02:31:03 dist-test-slave-dn4n krb5kdc[1695](info): commencing operation
krb5kdc: starting...
W20260925 02:31:04.033571 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:31:04 dist-test-slave-dn4n krb5kdc[1695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:31:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:31:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:31:04.133002 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:45997
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:45997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:31:04.205892 1709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:04.206079 1709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:04.206194 1709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:04.207268 1709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:04.207304 1709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:04.207316 1709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:04.207360 1709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:04.207381 1709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:04.208745 1709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:45997
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:45997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:04.208974 1709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:04.209174 1709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:04.213394 1714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:04.213433 1715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:04.213665 1709 server_base.cc:1034] running on GCE node
W20260925 02:31:04.213400 1717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:04.213979 1709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:04.214268 1709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:04.215431 1709 hybrid_clock.cc:648] HybridClock initialized: now 1790303464215413 us; error 46 us; skew 500 ppm
Sep 25 02:31:04 dist-test-slave-dn4n krb5kdc[1695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303464, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:04.217856 1709 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:31:04.218298 1709 webserver.cc:460] Webserver started at http://127.11.220.254:37875/ using document root <none> and password file <none>
I20260925 02:31:04.218511 1709 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:04.218572 1709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:04.218693 1709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:04.219626 1709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "2d637e3c10d343988553b80f9ae82cd3"
format_stamp: "Formatted at 2026-09-25 02:31:04 on dist-test-slave-dn4n"
server_key: "cfcf9f2f2c02d10c18f62a5ab15f771d"
server_key_iv: "9b6b7c2fb8624c7c694718c278046a1a"
server_key_version: "encryptionkey@0"
I20260925 02:31:04.219967 1709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "2d637e3c10d343988553b80f9ae82cd3"
format_stamp: "Formatted at 2026-09-25 02:31:04 on dist-test-slave-dn4n"
server_key: "cfcf9f2f2c02d10c18f62a5ab15f771d"
server_key_iv: "9b6b7c2fb8624c7c694718c278046a1a"
server_key_version: "encryptionkey@0"
I20260925 02:31:04.221253 1709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:31:04.221915 1724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:04.222078 1709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:31:04.222172 1709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "2d637e3c10d343988553b80f9ae82cd3"
format_stamp: "Formatted at 2026-09-25 02:31:04 on dist-test-slave-dn4n"
server_key: "cfcf9f2f2c02d10c18f62a5ab15f771d"
server_key_iv: "9b6b7c2fb8624c7c694718c278046a1a"
server_key_version: "encryptionkey@0"
I20260925 02:31:04.222244 1709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:04.230989 1709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:04.233018 1709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:04.233109 1709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:04.236740 1709 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:45997
I20260925 02:31:04.236783 1777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:45997 every 8 connection(s)
I20260925 02:31:04.237063 1709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:31:04.237771 1778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:04.239696 1778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3: Bootstrap starting.
I20260925 02:31:04.240221 1778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:04.240530 1778 log.cc:824] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:04.241147 1778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3: No bootstrap required, opened a new log
I20260925 02:31:04.242700 1778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d637e3c10d343988553b80f9ae82cd3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45997 } }
I20260925 02:31:04.242822 1778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:31:04.242842 1778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d637e3c10d343988553b80f9ae82cd3, State: Initialized, Role: FOLLOWER
I20260925 02:31:04.243034 1778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d637e3c10d343988553b80f9ae82cd3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45997 } }
I20260925 02:31:04.243113 1778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:31:04.243151 1778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:31:04.243199 1778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:04.244006 1778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d637e3c10d343988553b80f9ae82cd3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45997 } }
I20260925 02:31:04.244117 1778 leader_election.cc:302] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d637e3c10d343988553b80f9ae82cd3; no voters:
I20260925 02:31:04.244293 1778 leader_election.cc:288] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:31:04.244326 1781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:31:04.244568 1778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:04.244623 1781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [term 1 LEADER]: Becoming Leader. State: Replica: 2d637e3c10d343988553b80f9ae82cd3, State: Running, Role: LEADER
I20260925 02:31:04.244766 1781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d637e3c10d343988553b80f9ae82cd3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45997 } }
I20260925 02:31:04.245199 1782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d637e3c10d343988553b80f9ae82cd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d637e3c10d343988553b80f9ae82cd3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45997 } } }
I20260925 02:31:04.245285 1782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:04.245483 1789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:31:04.245507 1783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d637e3c10d343988553b80f9ae82cd3. Latest consensus state: current_term: 1 leader_uuid: "2d637e3c10d343988553b80f9ae82cd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d637e3c10d343988553b80f9ae82cd3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45997 } } }
I20260925 02:31:04.245870 1783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:04.246009 1789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:31:04.246798 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 1709
I20260925 02:31:04.246913 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:31:04.247112 12147 external_mini_cluster.cc:1428] Setting key e5e5b5050628fb2632dc00709b755d37
I20260925 02:31:04.248703 1789 catalog_manager.cc:1347] Generated new cluster ID: 230c91a3891d4bb9b81b6c747a2dfa34
I20260925 02:31:04.248759 1789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:31:04.252065 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:04.247680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45003 (local address 127.11.220.254:45997)
0925 02:31:04.247847 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:04.247856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:04.247885 (+ 29us) server_negotiation.cc:407] Connection header received
0925 02:31:04.247992 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:04.247999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:04.248053 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:04.248191 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:04.248386 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:04.249492 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:04.250237 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:04.250657 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:04.250765 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:04.251420 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:04.251424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:04.251424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:04.251592 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:04.251622 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:04.251631 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:04.251722 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:04.251800 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 02:31:04.251917 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260925 02:31:04.252914 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:31:04.259186 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:04.254232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54883 (local address 127.11.220.254:45997)
0925 02:31:04.254266 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:04.254268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:04.256677 (+ 2409us) server_negotiation.cc:407] Connection header received
0925 02:31:04.256736 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:04.256737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:04.256747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:04.256795 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:04.256894 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:04.257369 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:04.258052 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:04.258259 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:04.258336 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:04.258945 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:04.258946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:04.258946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:04.258950 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:04.258963 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:04.258966 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:04.259024 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:04.259082 (+ 58us) server_negotiation.cc:299] Negotiation successful
0925 02:31:04.259093 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:31:04.266906 1789 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:31:04.267370 1789 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:31:04.273233 1789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d637e3c10d343988553b80f9ae82cd3: Generated new TSK 0
I20260925 02:31:04.273401 1789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:31:04.278095 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 1709
W20260925 02:31:04.283049 1693 connection.cc:535] client connection to 127.11.220.254:45997 recv error: Network error: failed to read from TLS socket (remote: 127.11.220.254:45997): Connection reset by peer (error 104)
2026-09-25T02:31:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1214 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:31:04 dist-test-slave-dn4n krb5kdc[1809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:31:04 dist-test-slave-dn4n krb5kdc[1809](info): set up 2 sockets
Sep 25 02:31:04 dist-test-slave-dn4n krb5kdc[1809](info): commencing operation
krb5kdc: starting...
W20260925 02:31:05.216840 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:31:05 dist-test-slave-dn4n krb5kdc[1809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:31:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:31:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:31:05.317394 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:42239
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:42239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:31:05.390466 1822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:05.390622 1822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:05.390642 1822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:05.391608 1822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:05.391637 1822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:05.391666 1822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:05.391677 1822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:05.391688 1822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:05.393054 1822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:42239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:42239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:05.393214 1822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:05.393352 1822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:05.397598 1830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:05.397738 1822 server_base.cc:1034] running on GCE node
W20260925 02:31:05.397652 1828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:05.397614 1827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:05.398149 1822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:05.398367 1822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:05.399498 1822 hybrid_clock.cc:648] HybridClock initialized: now 1790303465399479 us; error 34 us; skew 500 ppm
Sep 25 02:31:05 dist-test-slave-dn4n krb5kdc[1809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303465, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:05.401706 1822 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:31:05.402094 1822 webserver.cc:460] Webserver started at http://127.11.220.254:38665/ using document root <none> and password file <none>
I20260925 02:31:05.402281 1822 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:05.402316 1822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:05.402438 1822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:05.403354 1822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "44f7c53de2c34aacb78b647040355dc2"
format_stamp: "Formatted at 2026-09-25 02:31:05 on dist-test-slave-dn4n"
server_key: "6888f88c93a56cbacf81049e87274832"
server_key_iv: "5241fc9d8f2f68b506426ad65cc01fb5"
server_key_version: "encryptionkey@0"
I20260925 02:31:05.403672 1822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "44f7c53de2c34aacb78b647040355dc2"
format_stamp: "Formatted at 2026-09-25 02:31:05 on dist-test-slave-dn4n"
server_key: "6888f88c93a56cbacf81049e87274832"
server_key_iv: "5241fc9d8f2f68b506426ad65cc01fb5"
server_key_version: "encryptionkey@0"
I20260925 02:31:05.404896 1822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:31:05.405514 1837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:05.405658 1822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:31:05.405745 1822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "44f7c53de2c34aacb78b647040355dc2"
format_stamp: "Formatted at 2026-09-25 02:31:05 on dist-test-slave-dn4n"
server_key: "6888f88c93a56cbacf81049e87274832"
server_key_iv: "5241fc9d8f2f68b506426ad65cc01fb5"
server_key_version: "encryptionkey@0"
I20260925 02:31:05.405815 1822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:05.408607 1822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:05.410133 1822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:05.410277 1822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:05.413938 1822 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:42239
I20260925 02:31:05.413964 1889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:42239 every 8 connection(s)
I20260925 02:31:05.414235 1822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:31:05.415048 1890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:05.417008 1890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2: Bootstrap starting.
I20260925 02:31:05.417459 1890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:05.417675 1890 log.cc:824] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:05.418202 1890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2: No bootstrap required, opened a new log
I20260925 02:31:05.419546 1890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f7c53de2c34aacb78b647040355dc2" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42239 } }
I20260925 02:31:05.419654 1890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:31:05.419673 1890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f7c53de2c34aacb78b647040355dc2, State: Initialized, Role: FOLLOWER
I20260925 02:31:05.419783 1890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f7c53de2c34aacb78b647040355dc2" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42239 } }
I20260925 02:31:05.419853 1890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:31:05.419873 1890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:31:05.419898 1890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:05.420689 1890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f7c53de2c34aacb78b647040355dc2" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42239 } }
I20260925 02:31:05.420769 1890 leader_election.cc:302] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44f7c53de2c34aacb78b647040355dc2; no voters:
I20260925 02:31:05.420820 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 1822
I20260925 02:31:05.420956 1890 leader_election.cc:288] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:31:05.420960 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:31:05.421010 1893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:31:05.421159 12147 external_mini_cluster.cc:1428] Setting key 42a2d2a6b98f4690e5ab2eb4ad0d6218
I20260925 02:31:05.421221 1890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:05.421233 1893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [term 1 LEADER]: Becoming Leader. State: Replica: 44f7c53de2c34aacb78b647040355dc2, State: Running, Role: LEADER
I20260925 02:31:05.421437 1893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f7c53de2c34aacb78b647040355dc2" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42239 } }
I20260925 02:31:05.422000 1894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44f7c53de2c34aacb78b647040355dc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f7c53de2c34aacb78b647040355dc2" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42239 } } }
I20260925 02:31:05.422083 1894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:05.422006 1895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f7c53de2c34aacb78b647040355dc2. Latest consensus state: current_term: 1 leader_uuid: "44f7c53de2c34aacb78b647040355dc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f7c53de2c34aacb78b647040355dc2" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42239 } } }
I20260925 02:31:05.422123 1895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:05.422323 1904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:31:05.422847 1904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:31:05.423491 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:05.421653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39915 (local address 127.11.220.254:42239)
0925 02:31:05.421969 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:05.421977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:05.422010 (+ 33us) server_negotiation.cc:407] Connection header received
0925 02:31:05.422113 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:05.422119 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:05.422148 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:05.422558 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:05.422970 (+ 412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:05.422973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:05.422973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:05.422977 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:05.423013 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:05.423026 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:05.423273 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:05.423302 (+ 29us) server_negotiation.cc:299] Negotiation successful
0925 02:31:05.423337 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":57,"threads_started":1}
I20260925 02:31:05.424271 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:31:05.425499 1904 catalog_manager.cc:1347] Generated new cluster ID: 026074bb3b5d4659aeaec4c823741aca
I20260925 02:31:05.425552 1904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:31:05.429535 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:05.425620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45569 (local address 127.11.220.254:42239)
0925 02:31:05.425650 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:05.425651 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:05.428664 (+ 3013us) server_negotiation.cc:407] Connection header received
0925 02:31:05.428741 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:05.428742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:05.428747 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:05.428798 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:05.429338 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:05.429339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:05.429339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:05.429343 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:05.429355 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:05.429359 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:05.429409 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:05.429459 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 02:31:05.429470 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:31:05.443828 1904 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:31:05.443976 1904 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:31:05.456954 1904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44f7c53de2c34aacb78b647040355dc2: Generated new TSK 0
I20260925 02:31:05.457104 1904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:31:05.467955 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 1822
2026-09-25T02:31:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1188 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:31:05 dist-test-slave-dn4n krb5kdc[1921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:31:05 dist-test-slave-dn4n krb5kdc[1921](info): set up 2 sockets
Sep 25 02:31:05 dist-test-slave-dn4n krb5kdc[1921](info): commencing operation
krb5kdc: starting...
W20260925 02:31:06.417258 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:31:06 dist-test-slave-dn4n krb5kdc[1921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:31:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:31:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:31:06.516837 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:40977
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:40977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:31:06.589980 1935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:06.590157 1935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:06.590178 1935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:06.591236 1935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:06.591272 1935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:06.591285 1935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:06.591295 1935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:06.591341 1935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:06.592691 1935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:40977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:40977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:06.592851 1935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:06.593060 1935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:06.597568 1941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:06.597642 1940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:06.597568 1944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:06.597692 1935 server_base.cc:1034] running on GCE node
I20260925 02:31:06.597970 1935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:06.598220 1935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:06.599370 1935 hybrid_clock.cc:648] HybridClock initialized: now 1790303466599354 us; error 46 us; skew 500 ppm
Sep 25 02:31:06 dist-test-slave-dn4n krb5kdc[1921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303466, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:06.608639 1935 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:31:06.609077 1935 webserver.cc:460] Webserver started at http://127.11.220.254:39337/ using document root <none> and password file <none>
I20260925 02:31:06.609298 1935 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:06.609361 1935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:06.609498 1935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:06.610424 1935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "8b7b454072354376b78a95787e86e1e1"
format_stamp: "Formatted at 2026-09-25 02:31:06 on dist-test-slave-dn4n"
server_key: "712e38241ee583ebadeab31573e5ac70"
server_key_iv: "68810001f9cf69b83aca9b9189bd67a0"
server_key_version: "encryptionkey@0"
I20260925 02:31:06.610762 1935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "8b7b454072354376b78a95787e86e1e1"
format_stamp: "Formatted at 2026-09-25 02:31:06 on dist-test-slave-dn4n"
server_key: "712e38241ee583ebadeab31573e5ac70"
server_key_iv: "68810001f9cf69b83aca9b9189bd67a0"
server_key_version: "encryptionkey@0"
I20260925 02:31:06.612008 1935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 02:31:06.612725 1951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:06.612884 1935 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:31:06.612970 1935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "8b7b454072354376b78a95787e86e1e1"
format_stamp: "Formatted at 2026-09-25 02:31:06 on dist-test-slave-dn4n"
server_key: "712e38241ee583ebadeab31573e5ac70"
server_key_iv: "68810001f9cf69b83aca9b9189bd67a0"
server_key_version: "encryptionkey@0"
I20260925 02:31:06.613044 1935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:06.620520 1935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:06.622053 1935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:06.622198 1935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:06.625926 1935 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:40977
I20260925 02:31:06.625980 2005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:40977 every 8 connection(s)
I20260925 02:31:06.626284 1935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:31:06.627038 2006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:06.629071 2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1: Bootstrap starting.
I20260925 02:31:06.629535 2006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:06.629742 2006 log.cc:824] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:06.630334 2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1: No bootstrap required, opened a new log
I20260925 02:31:06.630971 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 1935
I20260925 02:31:06.631089 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:31:06.631269 12147 external_mini_cluster.cc:1428] Setting key 5b04120e34cfa9c187c0993f59cf865a
I20260925 02:31:06.632359 2006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7b454072354376b78a95787e86e1e1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40977 } }
I20260925 02:31:06.632531 2006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:31:06.632592 2006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b7b454072354376b78a95787e86e1e1, State: Initialized, Role: FOLLOWER
I20260925 02:31:06.632725 2006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7b454072354376b78a95787e86e1e1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40977 } }
I20260925 02:31:06.632805 2006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:31:06.632835 2006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:31:06.632874 2006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:06.633697 2006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7b454072354376b78a95787e86e1e1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40977 } }
I20260925 02:31:06.633780 2006 leader_election.cc:302] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b7b454072354376b78a95787e86e1e1; no voters:
I20260925 02:31:06.634006 2006 leader_election.cc:288] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:31:06.634027 2011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:31:06.634137 2011 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [term 1 LEADER]: Becoming Leader. State: Replica: 8b7b454072354376b78a95787e86e1e1, State: Running, Role: LEADER
I20260925 02:31:06.634260 2006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:06.634263 2011 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7b454072354376b78a95787e86e1e1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40977 } }
I20260925 02:31:06.634693 2012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b7b454072354376b78a95787e86e1e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7b454072354376b78a95787e86e1e1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40977 } } }
I20260925 02:31:06.634799 2012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:06.634704 2013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b7b454072354376b78a95787e86e1e1. Latest consensus state: current_term: 1 leader_uuid: "8b7b454072354376b78a95787e86e1e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7b454072354376b78a95787e86e1e1" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 40977 } } }
I20260925 02:31:06.634838 2013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:06.634999 2018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:31:06.635571 2018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:31:06.637128 2018 catalog_manager.cc:1347] Generated new cluster ID: d1cda9104c434698b70fcdbf1a59b66e
I20260925 02:31:06.637197 2018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 02:31:06 dist-test-slave-dn4n krb5kdc[1921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:31:06.643239 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:06.631717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54773 (local address 127.11.220.254:40977)
0925 02:31:06.631873 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:06.631880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:06.631901 (+ 21us) server_negotiation.cc:407] Connection header received
0925 02:31:06.632339 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:06.632345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:06.632384 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:06.632570 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:06.633280 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:06.634298 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:06.635368 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:06.635981 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:06.636097 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:06.638476 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:06.638482 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:06.638482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:06.638496 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:06.641848 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:06.642196 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:06.642199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:06.642199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:06.642306 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:06.642516 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:06.642517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:06.642518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:06.642721 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:06.642874 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:06.642943 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 02:31:06.643079 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260925 02:31:06.644100 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:31:06.648000 2018 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:31:06.648553 2018 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:31:06.652675 2018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b7b454072354376b78a95787e86e1e1: Generated new TSK 0
I20260925 02:31:06.652841 2018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:31:06.654909 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:06.645542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50795 (local address 127.11.220.254:40977)
0925 02:31:06.645575 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:06.645577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:06.648615 (+ 3038us) server_negotiation.cc:407] Connection header received
0925 02:31:06.648697 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:06.648698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:06.648708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:06.648764 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:06.649372 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:06.650066 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:06.651047 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:06.651345 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:06.651406 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:06.652628 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:06.652629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:06.652629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:06.652636 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:06.653904 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:06.654313 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:06.654313 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:06.654314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:06.654357 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:06.654538 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:06.654539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:06.654539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:06.654619 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:06.654682 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:06.654809 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 02:31:06.654860 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:31:06.656589 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 1935
2026-09-25T02:31:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1188 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:31:06 dist-test-slave-dn4n krb5kdc[2037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:31:06 dist-test-slave-dn4n krb5kdc[2037](info): set up 2 sockets
Sep 25 02:31:06 dist-test-slave-dn4n krb5kdc[2037](info): commencing operation
krb5kdc: starting...
W20260925 02:31:07.617354 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.943s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:31:07 dist-test-slave-dn4n krb5kdc[2037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:31:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:31:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:31:07.720206 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:45807
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:39705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:45807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:31:07.794062 2050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:07.794232 2050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:07.794255 2050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:07.795284 2050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:07.795315 2050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:07.795326 2050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:07.795337 2050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:07.795348 2050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:07.796661 2050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:39705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:45807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:45807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:07.796836 2050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:07.796978 2050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:07.801262 2056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:07.801291 2057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:07.801422 2059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:07.801430 2050 server_base.cc:1034] running on GCE node
I20260925 02:31:07.801697 2050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:07.801936 2050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:07.803085 2050 hybrid_clock.cc:648] HybridClock initialized: now 1790303467803082 us; error 30 us; skew 500 ppm
Sep 25 02:31:07 dist-test-slave-dn4n krb5kdc[2037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303467, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:07.805244 2050 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:31:07.805665 2050 webserver.cc:460] Webserver started at http://127.11.220.254:38525/ using document root <none> and password file <none>
I20260925 02:31:07.805867 2050 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:07.805939 2050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:07.806074 2050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:07.806941 2050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "dfc15c3765444a768b66dabe0077f492"
format_stamp: "Formatted at 2026-09-25 02:31:07 on dist-test-slave-dn4n"
server_key: "ff292d4a9e5f51dea44a7a7e86b0e5f3"
server_key_iv: "6b06d4c8f85a0acd4503224e4f2f4b2c"
server_key_version: "encryptionkey@0"
I20260925 02:31:07.807255 2050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "dfc15c3765444a768b66dabe0077f492"
format_stamp: "Formatted at 2026-09-25 02:31:07 on dist-test-slave-dn4n"
server_key: "ff292d4a9e5f51dea44a7a7e86b0e5f3"
server_key_iv: "6b06d4c8f85a0acd4503224e4f2f4b2c"
server_key_version: "encryptionkey@0"
I20260925 02:31:07.808432 2050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 02:31:07.809129 2067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:07.809257 2050 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:31:07.809329 2050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "dfc15c3765444a768b66dabe0077f492"
format_stamp: "Formatted at 2026-09-25 02:31:07 on dist-test-slave-dn4n"
server_key: "ff292d4a9e5f51dea44a7a7e86b0e5f3"
server_key_iv: "6b06d4c8f85a0acd4503224e4f2f4b2c"
server_key_version: "encryptionkey@0"
I20260925 02:31:07.809396 2050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:07.828874 2050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:07.830892 2050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:07.831004 2050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:07.834725 2050 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:45807
I20260925 02:31:07.834754 2119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:45807 every 8 connection(s)
I20260925 02:31:07.835059 2050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:31:07.835819 2120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:07.837795 2120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492: Bootstrap starting.
I20260925 02:31:07.838299 2120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:07.838557 2120 log.cc:824] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:07.839291 2120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492: No bootstrap required, opened a new log
I20260925 02:31:07.840838 2120 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc15c3765444a768b66dabe0077f492" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45807 } }
I20260925 02:31:07.840969 2120 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:31:07.840996 2120 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc15c3765444a768b66dabe0077f492, State: Initialized, Role: FOLLOWER
I20260925 02:31:07.841166 2120 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc15c3765444a768b66dabe0077f492" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45807 } }
I20260925 02:31:07.841243 2120 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:31:07.841292 2120 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:31:07.841338 2120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:07.842043 2120 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc15c3765444a768b66dabe0077f492" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45807 } }
I20260925 02:31:07.842142 2120 leader_election.cc:302] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfc15c3765444a768b66dabe0077f492; no voters:
I20260925 02:31:07.842303 2120 leader_election.cc:288] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:31:07.842337 2123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:31:07.842474 2123 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [term 1 LEADER]: Becoming Leader. State: Replica: dfc15c3765444a768b66dabe0077f492, State: Running, Role: LEADER
I20260925 02:31:07.842525 2120 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:07.842589 2123 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc15c3765444a768b66dabe0077f492" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45807 } }
I20260925 02:31:07.842988 2124 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfc15c3765444a768b66dabe0077f492" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc15c3765444a768b66dabe0077f492" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45807 } } }
I20260925 02:31:07.843000 2125 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfc15c3765444a768b66dabe0077f492. Latest consensus state: current_term: 1 leader_uuid: "dfc15c3765444a768b66dabe0077f492" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfc15c3765444a768b66dabe0077f492" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 45807 } } }
I20260925 02:31:07.843102 2124 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:07.843117 2125 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:07.843741 2133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:31:07.844246 2133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:31:07.844307 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 2050
I20260925 02:31:07.844415 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:31:07.844748 12147 external_mini_cluster.cc:1428] Setting key d5030760b4757bf48e605054ac9acfd9
I20260925 02:31:07.846213 2133 catalog_manager.cc:1347] Generated new cluster ID: 86f5a50fb83e4efdb5d75b8c69ed3b77
I20260925 02:31:07.846257 2133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:31:07.852870 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:07.845283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46091 (local address 127.11.220.254:45807)
0925 02:31:07.845519 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:07.845526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:07.845556 (+ 30us) server_negotiation.cc:407] Connection header received
0925 02:31:07.845660 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:07.845665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:07.845712 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:07.845834 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:07.846034 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:07.849941 (+ 3907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:07.850978 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:07.851514 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:07.851601 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:07.852225 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:07.852228 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:07.852229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:07.852414 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:07.852446 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:07.852481 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:07.852547 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:07.852629 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 02:31:07.852681 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
I20260925 02:31:07.853710 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:31:07.856782 2133 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:31:07.857214 2133 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:31:07.858413 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:07.855192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43701 (local address 127.11.220.254:45807)
0925 02:31:07.855221 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:07.855222 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:07.855230 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:31:07.855285 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:07.855286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:07.855294 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:07.855352 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:07.855463 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:07.856150 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:07.857275 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:07.857534 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:07.857595 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:07.858184 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:07.858186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:07.858186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:07.858191 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:07.858203 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:07.858207 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:07.858259 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:07.858315 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 02:31:07.858348 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 02:31:07.859699 2133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfc15c3765444a768b66dabe0077f492: Generated new TSK 0
I20260925 02:31:07.859843 2133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:31:07.863571 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 2050
2026-09-25T02:31:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1207 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:31:07 dist-test-slave-dn4n krb5kdc[2153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:31:07 dist-test-slave-dn4n krb5kdc[2153](info): set up 2 sockets
Sep 25 02:31:07 dist-test-slave-dn4n krb5kdc[2153](info): commencing operation
krb5kdc: starting...
W20260925 02:31:08.845629 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:31:08 dist-test-slave-dn4n krb5kdc[2153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:31:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:31:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:31:08.947013 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:37429
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:44247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:37429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:31:09.021328 2166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:09.021539 2166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:09.021574 2166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:09.022611 2166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:09.022677 2166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:09.022697 2166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:09.022716 2166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:09.022743 2166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:09.024096 2166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:44247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:37429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:37429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:09.024348 2166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:09.024573 2166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:09.034646 2171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:09.034653 2172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:09.034761 2174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:09.035838 2166 server_base.cc:1034] running on GCE node
I20260925 02:31:09.036078 2166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:09.036367 2166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:09.037552 2166 hybrid_clock.cc:648] HybridClock initialized: now 1790303469037506 us; error 43 us; skew 500 ppm
Sep 25 02:31:09 dist-test-slave-dn4n krb5kdc[2153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303469, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:09.039988 2166 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:31:09.040385 2166 webserver.cc:460] Webserver started at http://127.11.220.254:43045/ using document root <none> and password file <none>
I20260925 02:31:09.040599 2166 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:09.040640 2166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:09.040721 2166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:09.041625 2166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "0cf11b137ee94f6ab5c2b58cee9250a7"
format_stamp: "Formatted at 2026-09-25 02:31:09 on dist-test-slave-dn4n"
server_key: "19823ecff8b998007e4c4667c7a11384"
server_key_iv: "5850788d447a3cc1e7e5a2879d6d0352"
server_key_version: "encryptionkey@0"
I20260925 02:31:09.041956 2166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "0cf11b137ee94f6ab5c2b58cee9250a7"
format_stamp: "Formatted at 2026-09-25 02:31:09 on dist-test-slave-dn4n"
server_key: "19823ecff8b998007e4c4667c7a11384"
server_key_iv: "5850788d447a3cc1e7e5a2879d6d0352"
server_key_version: "encryptionkey@0"
I20260925 02:31:09.043190 2166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:31:09.043769 2181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:09.043907 2166 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:31:09.043983 2166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "0cf11b137ee94f6ab5c2b58cee9250a7"
format_stamp: "Formatted at 2026-09-25 02:31:09 on dist-test-slave-dn4n"
server_key: "19823ecff8b998007e4c4667c7a11384"
server_key_iv: "5850788d447a3cc1e7e5a2879d6d0352"
server_key_version: "encryptionkey@0"
I20260925 02:31:09.044075 2166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:09.046945 2166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:09.048398 2166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:09.048552 2166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:09.051829 2166 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:37429
I20260925 02:31:09.051870 2233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:37429 every 8 connection(s)
I20260925 02:31:09.052073 2166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:31:09.052975 2234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:09.054369 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 2166
I20260925 02:31:09.054489 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:31:09.054672 12147 external_mini_cluster.cc:1428] Setting key 33a814e5d293b22a54666c4ded8b39ae
I20260925 02:31:09.055389 2234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7: Bootstrap starting.
I20260925 02:31:09.056007 2234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:09.056335 2234 log.cc:824] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:09.057111 2234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7: No bootstrap required, opened a new log
I20260925 02:31:09.057114 2237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:09.055139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41979 (local address 127.11.220.254:37429)
0925 02:31:09.055339 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:09.055347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:09.055372 (+ 25us) server_negotiation.cc:407] Connection header received
0925 02:31:09.055881 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:09.055888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:09.055919 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:09.056059 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:09.056711 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:09.056713 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:09.056714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:09.056717 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:09.056749 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:09.056758 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:09.056855 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:09.056919 (+ 64us) server_negotiation.cc:299] Negotiation successful
0925 02:31:09.056956 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":63,"threads_started":1}
I20260925 02:31:09.057922 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:31:09.059077 2234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf11b137ee94f6ab5c2b58cee9250a7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37429 } }
I20260925 02:31:09.059233 2234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 02:31:09.059278 2234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf11b137ee94f6ab5c2b58cee9250a7, State: Initialized, Role: FOLLOWER
I20260925 02:31:09.059391 2234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf11b137ee94f6ab5c2b58cee9250a7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37429 } }
I20260925 02:31:09.059484 2234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 02:31:09.059525 2234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 02:31:09.059584 2234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:09.060178 2237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:09.059133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50409 (local address 127.11.220.254:37429)
0925 02:31:09.059166 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:09.059168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:09.059281 (+ 113us) server_negotiation.cc:407] Connection header received
0925 02:31:09.059361 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:09.059362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:09.059366 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:09.059419 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:09.059976 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:09.059977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:09.059977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 02:31:09.059981 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:09.059991 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 02:31:09.059994 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:09.060037 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:09.060089 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 02:31:09.060099 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:31:09.060353 2234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf11b137ee94f6ab5c2b58cee9250a7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37429 } }
I20260925 02:31:09.060496 2234 leader_election.cc:302] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cf11b137ee94f6ab5c2b58cee9250a7; no voters:
I20260925 02:31:09.060689 2234 leader_election.cc:288] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 02:31:09.060725 2244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:31:09.060886 2244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [term 1 LEADER]: Becoming Leader. State: Replica: 0cf11b137ee94f6ab5c2b58cee9250a7, State: Running, Role: LEADER
I20260925 02:31:09.060936 2234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:09.061026 2244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf11b137ee94f6ab5c2b58cee9250a7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37429 } }
I20260925 02:31:09.061394 2245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cf11b137ee94f6ab5c2b58cee9250a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf11b137ee94f6ab5c2b58cee9250a7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37429 } } }
I20260925 02:31:09.061404 2246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cf11b137ee94f6ab5c2b58cee9250a7. Latest consensus state: current_term: 1 leader_uuid: "0cf11b137ee94f6ab5c2b58cee9250a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf11b137ee94f6ab5c2b58cee9250a7" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 37429 } } }
I20260925 02:31:09.061524 2245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:09.061539 2246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7 [sys.catalog]: This master's current role is: LEADER
W20260925 02:31:09.062577 2260 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:31:09.062633 2260 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:31:09.062683 2253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:31:09.062846 2253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:31:09.064141 2253 catalog_manager.cc:1347] Generated new cluster ID: 98bf585e505444b1a5009c9ca2daf5d9
I20260925 02:31:09.064181 2253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:31:09.067647 2253 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:31:09.067802 2253 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:31:09.077797 2253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cf11b137ee94f6ab5c2b58cee9250a7: Generated new TSK 0
I20260925 02:31:09.077960 2253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 02:31:09.080137 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 2166
2026-09-25T02:31:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1216 ms)
[----------] 18 tests from AuthTokenIssuingTest (21745 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:31:09 dist-test-slave-dn4n krb5kdc[2265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:31:09 dist-test-slave-dn4n krb5kdc[2265](info): set up 2 sockets
Sep 25 02:31:09 dist-test-slave-dn4n krb5kdc[2265](info): commencing operation
krb5kdc: starting...
W20260925 02:31:10.045804 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.948s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:31:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:31:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:31:10.148880 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:35641
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:35641,127.11.220.253:33883,127.11.220.252:35587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:31:10.222954 2279 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:31:10.223138 2279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:10.223155 2279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:10.223174 2279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:10.224323 2279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:10.224359 2279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:10.224372 2279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:10.224416 2279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:10.224462 2279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:10.225931 2279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:35641,127.11.220.253:33883,127.11.220.252:35587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:35641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:10.226169 2279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:10.226368 2279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:10.230688 2285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:10.230666 2287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:10.230784 2279 server_base.cc:1034] running on GCE node
W20260925 02:31:10.230677 2284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:10.231060 2279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:10.231317 2279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:10.232483 2279 hybrid_clock.cc:648] HybridClock initialized: now 1790303470232438 us; error 25 us; skew 500 ppm
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:10.234668 2279 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:31:10.235106 2279 webserver.cc:460] Webserver started at http://127.11.220.254:41287/ using document root <none> and password file <none>
I20260925 02:31:10.235323 2279 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:10.235373 2279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:10.235520 2279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:10.236394 2279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3"
format_stamp: "Formatted at 2026-09-25 02:31:10 on dist-test-slave-dn4n"
server_key: "28762b379e7bfd002834e8576c694114"
server_key_iv: "e487216bdbee179160d7f18a7a4afcdd"
server_key_version: "encryptionkey@0"
I20260925 02:31:10.236783 2279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3"
format_stamp: "Formatted at 2026-09-25 02:31:10 on dist-test-slave-dn4n"
server_key: "28762b379e7bfd002834e8576c694114"
server_key_iv: "e487216bdbee179160d7f18a7a4afcdd"
server_key_version: "encryptionkey@0"
I20260925 02:31:10.237962 2279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:31:10.238651 2294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:10.238814 2279 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:31:10.238896 2279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3"
format_stamp: "Formatted at 2026-09-25 02:31:10 on dist-test-slave-dn4n"
server_key: "28762b379e7bfd002834e8576c694114"
server_key_iv: "e487216bdbee179160d7f18a7a4afcdd"
server_key_version: "encryptionkey@0"
I20260925 02:31:10.238945 2279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:10.271237 2279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:10.273253 2279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:10.273347 2279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:10.276921 2279 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:35641
I20260925 02:31:10.276973 2347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:35641 every 8 connection(s)
I20260925 02:31:10.277253 2279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:31:10.278018 2348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:10.279306 2348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:10.282754 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 2279
I20260925 02:31:10.282855 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:31:10.283013 12147 external_mini_cluster.cc:1428] Setting key 025c011db451d72a021ec27d46436b3e
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:31:10.287955 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.279868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35641 (local address 127.0.0.1:34312)
0925 02:31:10.280014 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.280046 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:10.280494 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:10.280745 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:10.280747 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:10.280978 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:10.281134 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.281138 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.282039 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.282040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.282617 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.282619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.283212 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.283213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.283264 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.283667 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:10.283673 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:10.284933 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:10.286682 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.286688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.286689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.286879 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.287138 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.287139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.287139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.287210 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.287610 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:10.287614 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:10.287659 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 02:31:10.287720 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 02:31:10.287823 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260925 02:31:10.288095 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.280018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.11.220.254:35641)
0925 02:31:10.280139 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.280143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.280169 (+ 26us) server_negotiation.cc:407] Connection header received
0925 02:31:10.280630 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.280632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.280648 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.280717 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.281205 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.281970 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.282687 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.283133 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.283237 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.285018 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.285025 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.285025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.285032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.286604 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.286963 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.286966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.286966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.287071 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.287271 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.287273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.287273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.287541 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.287627 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.287784 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.287905 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
I20260925 02:31:10.288985 2348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:10.289266 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.289169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:33883 (local address 127.0.0.1:37850)
0925 02:31:10.289210 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.289251 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 02:31:10.289417 2298 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.220.253:33883: connect: Connection refused (error 111)
I20260925 02:31:10.289804 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.289726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:33883 (local address 127.0.0.1:37858)
0925 02:31:10.289768 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.289789 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 02:31:10.289925 2348 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.253:33883: Network error: Client connection negotiation failed: client connection to 127.11.220.253:33883: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.220.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab.
Entry for principal kudu/127.11.220.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab.
I20260925 02:31:10.304781 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.253:33883
--webserver_interface=127.11.220.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab
--principal=kudu/127.11.220.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:35641,127.11.220.253:33883,127.11.220.252:35587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:31:10.339133 2348 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } attempt: 1
I20260925 02:31:10.339573 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.339442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:33883 (local address 127.0.0.1:37868)
0925 02:31:10.339507 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.339543 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 02:31:10.339926 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.339813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:33883 (local address 127.0.0.1:37884)
0925 02:31:10.339857 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.339880 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 02:31:10.340032 2348 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.253:33883: Network error: Client connection negotiation failed: client connection to 127.11.220.253:33883: connect: Connection refused (error 111)
W20260925 02:31:10.379231 2355 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:31:10.379436 2355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:10.379469 2355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:10.379506 2355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:10.380582 2355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:10.380641 2355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:10.380662 2355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:10.380687 2355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:10.380714 2355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:10.381982 2355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:35641,127.11.220.253:33883,127.11.220.252:35587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab
--principal=kudu/127.11.220.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.253:33883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.220.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:10.382227 2355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:10.382421 2355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:10.386605 2360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:10.386615 2361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:10.386652 2363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:10.386768 2355 server_base.cc:1034] running on GCE node
I20260925 02:31:10.387010 2355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:10.387251 2355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:10.388371 2355 hybrid_clock.cc:648] HybridClock initialized: now 1790303470388359 us; error 22 us; skew 500 ppm
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:10.390532 2355 init.cc:377] Logged in from keytab as kudu/127.11.220.253@KRBTEST.COM (short username kudu)
I20260925 02:31:10.390936 2355 webserver.cc:460] Webserver started at http://127.11.220.253:33455/ using document root <none> and password file <none>
I20260925 02:31:10.391103 2355 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:10.391139 2355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:10.391215 2355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:10.392043 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/data/instance:
uuid: "68c12ac78b2b4528b5eaf28f390f7b3d"
format_stamp: "Formatted at 2026-09-25 02:31:10 on dist-test-slave-dn4n"
server_key: "c9c5746b2e9707a44b25058ce75070e4"
server_key_iv: "a9b07bac56e829f7fb2fe0ad4baf0502"
server_key_version: "encryptionkey@0"
I20260925 02:31:10.392324 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/wal/instance:
uuid: "68c12ac78b2b4528b5eaf28f390f7b3d"
format_stamp: "Formatted at 2026-09-25 02:31:10 on dist-test-slave-dn4n"
server_key: "c9c5746b2e9707a44b25058ce75070e4"
server_key_iv: "a9b07bac56e829f7fb2fe0ad4baf0502"
server_key_version: "encryptionkey@0"
I20260925 02:31:10.393559 2355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 02:31:10.394137 2370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:10.394322 2355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 02:31:10.394410 2355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/wal
uuid: "68c12ac78b2b4528b5eaf28f390f7b3d"
format_stamp: "Formatted at 2026-09-25 02:31:10 on dist-test-slave-dn4n"
server_key: "c9c5746b2e9707a44b25058ce75070e4"
server_key_iv: "a9b07bac56e829f7fb2fe0ad4baf0502"
server_key_version: "encryptionkey@0"
I20260925 02:31:10.394486 2355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:10.408233 2348 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } attempt: 2
I20260925 02:31:10.408624 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.408498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:33883 (local address 127.0.0.1:37892)
0925 02:31:10.408549 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.408578 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 02:31:10.408913 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.408831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:33883 (local address 127.0.0.1:37902)
0925 02:31:10.408871 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.408891 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 02:31:10.408983 2348 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.253:33883: Network error: Client connection negotiation failed: client connection to 127.11.220.253:33883: connect: Connection refused (error 111)
I20260925 02:31:10.426451 2355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:10.428103 2355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:10.428246 2355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:10.431697 2355 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.253:33883
I20260925 02:31:10.431726 2423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.253:33883 every 8 connection(s)
I20260925 02:31:10.432022 2355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
I20260925 02:31:10.432822 2424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:10.434113 2424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:31:10.439059 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 2355
I20260925 02:31:10.439178 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-1/wal/instance
I20260925 02:31:10.439383 12147 external_mini_cluster.cc:1428] Setting key e3ef5e4104bd2d8e610f2fa6cd7a5ace
I20260925 02:31:10.442127 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.434674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.11.220.254:35641)
0925 02:31:10.434729 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.434733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.434904 (+ 171us) server_negotiation.cc:407] Connection header received
0925 02:31:10.435345 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.435346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.435357 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.435397 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.435873 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.436352 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.437221 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.437402 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.437450 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.439226 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.439227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.439228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.439235 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.440749 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.441240 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.441241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.441242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.441310 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.441552 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.441553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.441553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.441656 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.441725 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.441971 (+ 246us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.442046 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 02:31:10.442330 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.434649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35641 (local address 127.0.0.1:34322)
0925 02:31:10.434792 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.434829 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:10.435239 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:10.435450 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:10.435453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:10.435653 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:10.435812 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.435815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.436424 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.436425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.437156 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.437158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.437467 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.437468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.437560 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.437927 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:10.437934 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:10.439122 (+ 1188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:10.440848 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.440856 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.440859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.441169 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.441358 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.441359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.441360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.441473 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.441729 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:10.441736 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:10.441852 (+ 116us) client_negotiation.cc:769] Sending connection context
0925 02:31:10.441952 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 02:31:10.442099 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260925 02:31:10.442910 2424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.220.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.252@KRBTEST.COM" created.
I20260925 02:31:10.450356 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.443149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:33883 (local address 127.0.0.1:37912)
0925 02:31:10.443189 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.443198 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:10.443228 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:10.443447 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:10.443448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:10.443589 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:10.443644 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.443645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.444409 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.444410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.445311 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.445313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.445795 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.445797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.445876 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.446226 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:10.446231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:10.447520 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:10.449367 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.449368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.449368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.449524 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.449745 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.449745 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.449746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.449809 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.450063 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:10.450064 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:10.450098 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 02:31:10.450176 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 02:31:10.450261 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 02:31:10.450497 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.443223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37912 (local address 127.11.220.253:33883)
0925 02:31:10.443336 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.443339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.443352 (+ 13us) server_negotiation.cc:407] Connection header received
0925 02:31:10.443365 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.443366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.443374 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.443419 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.443712 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.444329 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.445416 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.445711 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.445776 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.447618 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.447632 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.447633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.447638 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.449274 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.449614 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.449617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.449617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.449693 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.449868 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.449869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.449869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.449987 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.450067 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.450206 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.450307 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":91,"thread_start_us":72,"threads_started":1}
I20260925 02:31:10.451031 2424 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:10.451345 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.451230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45470)
0925 02:31:10.451273 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.451319 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 02:31:10.451464 2373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
I20260925 02:31:10.451833 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.451752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45484)
0925 02:31:10.451792 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.451815 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 02:31:10.451952 2424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:35587: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab.
Entry for principal kudu/127.11.220.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab.
I20260925 02:31:10.463249 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.252:35587
--webserver_interface=127.11.220.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab
--principal=kudu/127.11.220.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:35641,127.11.220.253:33883,127.11.220.252:35587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:31:10.500109 2348 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } attempt: 3
I20260925 02:31:10.501150 2424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } attempt: 1
I20260925 02:31:10.501477 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.501357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45500)
0925 02:31:10.501414 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.501447 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 02:31:10.502704 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.502602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45508)
0925 02:31:10.502651 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.502680 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 02:31:10.502815 2424 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:35587: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
I20260925 02:31:10.507134 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.500498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.11.220.253:33883)
0925 02:31:10.500560 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.500562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.500572 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:31:10.500629 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.500630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.500641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.500708 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.500997 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.501588 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.502518 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.502773 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.502837 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.504690 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.504696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.504696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.504702 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.506065 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.506390 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.506391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.506391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.506452 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.506642 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.506643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.506643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.506738 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.506805 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.506988 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.507063 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 02:31:10.507141 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.500410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:33883 (local address 127.0.0.1:37918)
0925 02:31:10.500489 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.500507 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:10.500571 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:10.500714 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:10.500715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:10.500859 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:10.500915 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.500917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.501675 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.501677 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.502419 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.502422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.502861 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.502862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.502944 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.503301 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:10.503306 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:10.504597 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:10.506160 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.506161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.506162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.506321 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.506521 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.506521 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.506522 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.506575 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.506809 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:10.506810 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:10.506862 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 02:31:10.506965 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 02:31:10.507062 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 02:31:10.507453 2348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:10.507748 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.507667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45522)
0925 02:31:10.507710 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.507733 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 02:31:10.508006 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.507927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45534)
0925 02:31:10.507970 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.507989 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 02:31:10.508085 2348 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:35587: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
W20260925 02:31:10.538058 2432 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:31:10.538260 2432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:10.538292 2432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:10.538326 2432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 02:31:10.539203 2348 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } attempt: 1
W20260925 02:31:10.539474 2432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:10.539525 2432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:10.539547 2432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260925 02:31:10.539531 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.539425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45540)
0925 02:31:10.539487 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.539511 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 02:31:10.539577 2432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:10.539597 2432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:10.539861 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.539779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45554)
0925 02:31:10.539818 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.539839 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260925 02:31:10.539971 2348 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:35587: Network error: Client connection negotiation failed: client connection to 127.11.220.252:35587: connect: Connection refused (error 111)
I20260925 02:31:10.541194 2432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:35641,127.11.220.253:33883,127.11.220.252:35587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab
--principal=kudu/127.11.220.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.252:35587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.220.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:10.541415 2432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:10.541637 2432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:10.545851 2437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:10.545894 2438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:10.545958 2432 server_base.cc:1034] running on GCE node
W20260925 02:31:10.545856 2440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:10.546308 2432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:10.546542 2432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:10.547673 2432 hybrid_clock.cc:648] HybridClock initialized: now 1790303470547659 us; error 29 us; skew 500 ppm
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:10.549809 2432 init.cc:377] Logged in from keytab as kudu/127.11.220.252@KRBTEST.COM (short username kudu)
I20260925 02:31:10.550134 2432 webserver.cc:460] Webserver started at http://127.11.220.252:35539/ using document root <none> and password file <none>
I20260925 02:31:10.550287 2432 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:10.550320 2432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:10.550396 2432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:10.551258 2432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/data/instance:
uuid: "c30fe419207e45339059576a350ac3e9"
format_stamp: "Formatted at 2026-09-25 02:31:10 on dist-test-slave-dn4n"
server_key: "d4bf770b93bb198f1559c39eb51d8bcb"
server_key_iv: "0a81a9df6e2102ab1130619f506942d1"
server_key_version: "encryptionkey@0"
I20260925 02:31:10.551537 2432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/wal/instance:
uuid: "c30fe419207e45339059576a350ac3e9"
format_stamp: "Formatted at 2026-09-25 02:31:10 on dist-test-slave-dn4n"
server_key: "d4bf770b93bb198f1559c39eb51d8bcb"
server_key_iv: "0a81a9df6e2102ab1130619f506942d1"
server_key_version: "encryptionkey@0"
I20260925 02:31:10.552675 2432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:31:10.553292 2448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:10.553442 2432 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:31:10.553566 2432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/wal
uuid: "c30fe419207e45339059576a350ac3e9"
format_stamp: "Formatted at 2026-09-25 02:31:10 on dist-test-slave-dn4n"
server_key: "d4bf770b93bb198f1559c39eb51d8bcb"
server_key_iv: "0a81a9df6e2102ab1130619f506942d1"
server_key_version: "encryptionkey@0"
I20260925 02:31:10.553643 2432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:10.560326 2432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:10.562259 2432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:10.562417 2432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:10.566007 2432 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.252:35587
I20260925 02:31:10.566049 2501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.252:35587 every 8 connection(s)
I20260925 02:31:10.566370 2432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
I20260925 02:31:10.567004 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 2432
I20260925 02:31:10.567118 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790303222243611-12147-0/minicluster-data/master-2/wal/instance
I20260925 02:31:10.567283 12147 external_mini_cluster.cc:1428] Setting key fe955d21b99133a53f73e9b49f37a1e1
I20260925 02:31:10.567317 2502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:10.569603 2502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:10.570997 2424 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } attempt: 2
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
I20260925 02:31:10.577353 2505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.567758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45564 (local address 127.11.220.252:35587)
0925 02:31:10.567929 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.567935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.567957 (+ 22us) server_negotiation.cc:407] Connection header received
0925 02:31:10.568326 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.568331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.568371 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.568538 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.569298 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.570191 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.571010 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.571588 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.571675 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.574155 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.574163 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.574163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.574169 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.575714 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.576084 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.576087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.576087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.576178 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.576386 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.576387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.576387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.576692 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.576890 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.577007 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.577121 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260925 02:31:10.578922 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.570151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34326 (local address 127.11.220.254:35641)
0925 02:31:10.570212 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.570214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.570338 (+ 124us) server_negotiation.cc:407] Connection header received
0925 02:31:10.570374 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.570375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.570396 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.570439 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.570780 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.571393 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.572354 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.572626 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.572687 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.575952 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.575954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.575954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.575960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.577647 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.578131 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.578132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.578133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.578187 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.578399 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.578400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.578400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.578521 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.578570 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.578778 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.578847 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 02:31:10.578929 2506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.570077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:35641 (local address 127.0.0.1:34326)
0925 02:31:10.570237 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.570268 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:10.570327 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:10.570462 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:10.570463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:10.570659 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:10.570710 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.570711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.571464 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.571465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.572262 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.572264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.572710 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.572711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.572810 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.573811 (+ 1001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:10.573819 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:10.575894 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:10.577720 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.577726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.577729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.578026 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.578276 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.578277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.578278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.578331 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.578607 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:10.578612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:10.578647 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 02:31:10.578740 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 02:31:10.578835 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260925 02:31:10.579582 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.571205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45580)
0925 02:31:10.571379 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.571388 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:10.571423 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:10.571584 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:10.571585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:10.571734 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:10.571775 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.571777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.572577 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.572578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.573191 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.573194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.573651 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.573652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.573687 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.574062 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:10.574066 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:10.575815 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:10.578471 (+ 2656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.578472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.578473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.578719 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.578912 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.578913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.578913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.578973 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.579258 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:10.579259 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:10.579306 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 02:31:10.579353 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 02:31:10.579451 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260925 02:31:10.580029 2507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.571251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.11.220.252:35587)
0925 02:31:10.571401 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.571403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.571412 (+ 9us) server_negotiation.cc:407] Connection header received
0925 02:31:10.571486 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.571487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.571495 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.571562 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.571856 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.572505 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.573269 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.573580 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.573646 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.575908 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.575909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.575909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.575915 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.578400 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.578790 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.578791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.578791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.578842 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.579043 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.579044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.579044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.579184 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.579762 (+ 578us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.579800 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.579868 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260925 02:31:10.580243 2502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:10.581362 2424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d: Bootstrap starting.
I20260925 02:31:10.582305 2424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:10.582578 2424 log.cc:824] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:10.583264 2424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d: No bootstrap required, opened a new log
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:31:10.584887 2424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } }
I20260925 02:31:10.585116 2424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c12ac78b2b4528b5eaf28f390f7b3d, State: Initialized, Role: FOLLOWER
I20260925 02:31:10.585254 2424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } }
I20260925 02:31:10.585552 2510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } } }
I20260925 02:31:10.585618 2424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:10.585681 2510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:31:10.587525 2521 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:31:10.587597 2521 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
I20260925 02:31:10.587795 2508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.578366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34332 (local address 127.11.220.254:35641)
0925 02:31:10.578639 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.578642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.578648 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:31:10.578668 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.578669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.578678 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.580435 (+ 1757us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.580507 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.581279 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.582183 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.582356 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.582422 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.584500 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.584502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.584502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.584509 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.586222 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.586640 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.586641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.586641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.586698 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.586931 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.586932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.586932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.587075 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.587570 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.587605 (+ 35us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.587671 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":94,"threads_started":1}
I20260925 02:31:10.587814 2355 master_runner.cc:186] Error getting master registration for 127.11.220.254:35641: OK, instance_id { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" instance_seqno: 1790303470275869 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:31:10.590348 2355 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:31:10.590869 2506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.580437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:33883 (local address 127.0.0.1:37932)
0925 02:31:10.580480 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.580490 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:10.580527 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:10.584518 (+ 3991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:10.584519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:10.584654 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:10.584707 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.584708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.585424 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.585425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.586014 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.586016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.586469 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.586470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.586539 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.586910 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:10.586915 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:10.588081 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:10.589787 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.589788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.589789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.589982 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.590194 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.590195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.590195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.590256 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.590557 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:10.590558 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:10.590601 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 02:31:10.590665 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 02:31:10.590756 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 02:31:10.591310 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.582866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37932 (local address 127.11.220.253:33883)
0925 02:31:10.582913 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.582915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.582923 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:31:10.582941 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.582942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.582952 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.583014 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.584781 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.585328 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.586103 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.586402 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.586458 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.588154 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.588155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.588155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.588160 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.589685 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.590062 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.590063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.590063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.590122 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.590327 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.590327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.590328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.590474 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.590556 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.591144 (+ 588us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.591216 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 02:31:10.591593 2502 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
I20260925 02:31:10.598022 2506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.591828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45586)
0925 02:31:10.591859 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.591868 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:10.591903 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:10.592019 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:10.592020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:10.592156 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:10.592182 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.592183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.592987 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.592988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.593644 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.593646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.594051 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.594052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.594090 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.594445 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:10.594450 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:10.595501 (+ 1051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:10.597093 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.597093 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.597094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.597264 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.597468 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.597469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.597469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.597532 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.597755 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:10.597756 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:10.597797 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 02:31:10.597860 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 02:31:10.597958 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260925 02:31:10.598301 2507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.591869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45586 (local address 127.11.220.252:35587)
0925 02:31:10.591900 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.591902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.591909 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:31:10.591951 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.591951 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.591959 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.592014 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.592250 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.592908 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.593733 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.593970 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.594018 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.595578 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.595579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.595579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.595584 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.597016 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.597337 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.597338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.597338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.597396 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.597600 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.597601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.597601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.597700 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.597755 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.598172 (+ 417us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.598234 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:31:10.599155 2523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.590802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37948 (local address 127.11.220.253:33883)
0925 02:31:10.590942 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.590944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.590952 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:31:10.590981 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.590982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.590991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.591585 (+ 594us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.591614 (+ 29us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.592355 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.592981 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.593203 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.593279 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.595563 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.595564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.595564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.595570 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.597804 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.598298 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.598299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.598299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.598364 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.598575 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.598576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.598576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.598692 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.598827 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.598895 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.598964 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260925 02:31:10.599562 2502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9: Bootstrap starting.
I20260925 02:31:10.600292 2502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:10.600423 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:31:10.600643 2502 log.cc:824] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:10.601332 2502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9: No bootstrap required, opened a new log
I20260925 02:31:10.601464 2322 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34332
I20260925 02:31:10.602594 2502 raft_consensus.cc:348] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } }
I20260925 02:31:10.602684 2502 raft_consensus.cc:729] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c30fe419207e45339059576a350ac3e9, State: Initialized, Role: FOLLOWER
I20260925 02:31:10.602731 2502 consensus_queue.cc:260] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } }
I20260925 02:31:10.602934 2525 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } } }
I20260925 02:31:10.602982 2502 sys_catalog.cc:564] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:10.603006 2525 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:31:10.604933 2536 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:31:10.605017 2536 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:31:10.604912 2432 master_runner.cc:186] Error getting master registration for 127.11.220.254:35641: OK, instance_id { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" instance_seqno: 1790303470275869 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:31:10.606446 2432 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:31:10.615099 2348 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } attempt: 2
Sep 25 02:31:10 dist-test-slave-dn4n krb5kdc[2265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303470, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
I20260925 02:31:10.621969 2507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.615411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45600 (local address 127.11.220.252:35587)
0925 02:31:10.615460 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.615461 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.615468 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:31:10.615488 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.615488 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.615496 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.615585 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.615890 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.616318 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.617267 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.617433 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.617510 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.619376 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.619377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.619377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.619386 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.620986 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.621315 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.621316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.621316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.621361 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.621537 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.621538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.621538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.621639 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.621706 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.621829 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.621899 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:31:10.621969 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.615329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:35587 (local address 127.0.0.1:45600)
0925 02:31:10.615388 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:10.615398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:10.615432 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:10.615591 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:10.615592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:10.615779 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:10.615822 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.615824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.616397 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.616399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.617196 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:10.617198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.617512 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.617513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:10.617565 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.617964 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:10.617968 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:10.619308 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:10.621065 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.621066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.621066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.621252 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.621419 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:10.621419 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:10.621420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:10.621484 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:10.621701 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:10.621701 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:10.621741 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 02:31:10.621801 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 02:31:10.621887 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 02:31:10.623647 2348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3: Bootstrap starting.
I20260925 02:31:10.624308 2348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:10.624585 2348 log.cc:824] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:10.625291 2348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3: No bootstrap required, opened a new log
I20260925 02:31:10.626488 2348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } }
I20260925 02:31:10.626601 2348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8720d1a20e404a5ea8b6d3b50bc2a6e3, State: Initialized, Role: FOLLOWER
I20260925 02:31:10.626675 2348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } }
I20260925 02:31:10.626978 2322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 02:31:10.627036 2322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:10.627559 2322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } }
I20260925 02:31:10.627822 2322 leader_election.cc:288] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [CANDIDATE]: Term 1 election: Requested vote from peers 68c12ac78b2b4528b5eaf28f390f7b3d (127.11.220.253:33883), c30fe419207e45339059576a350ac3e9 (127.11.220.252:35587)
I20260925 02:31:10.626940 2348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:10.628100 2397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d"
I20260925 02:31:10.628144 2475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c30fe419207e45339059576a350ac3e9"
I20260925 02:31:10.628216 2397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:10.628262 2475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:10.629088 2397 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8720d1a20e404a5ea8b6d3b50bc2a6e3 in term 1.
I20260925 02:31:10.629240 2475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8720d1a20e404a5ea8b6d3b50bc2a6e3 in term 1.
I20260925 02:31:10.629460 2297 leader_election.cc:302] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8720d1a20e404a5ea8b6d3b50bc2a6e3, c30fe419207e45339059576a350ac3e9; no voters:
W20260925 02:31:10.630432 2555 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:31:10.630537 2555 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:31:10.632349 2279 master_runner.cc:263] No leader master found from master 8720d1a20e404a5ea8b6d3b50bc2a6e3
I20260925 02:31:10.632398 2279 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:31:10.626931 2539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } } }
I20260925 02:31:10.632488 2539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:31:10.632539 2539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:31:10.632589 2539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [term 1 LEADER]: Becoming Leader. State: Replica: 8720d1a20e404a5ea8b6d3b50bc2a6e3, State: Running, Role: LEADER
I20260925 02:31:10.632680 2539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } }
I20260925 02:31:10.633088 2539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8720d1a20e404a5ea8b6d3b50bc2a6e3. Latest consensus state: current_term: 1 leader_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } } }
I20260925 02:31:10.633169 2539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:10.636595 2559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:31:10.636768 2559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:31:10.642936 2475 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [term 1 FOLLOWER]: Refusing update from remote peer 8720d1a20e404a5ea8b6d3b50bc2a6e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:31:10.643316 2539 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:31:10.644786 2525 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8720d1a20e404a5ea8b6d3b50bc2a6e3. Latest consensus state: current_term: 1 leader_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } } }
I20260925 02:31:10.644871 2525 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:31:10.645217 2397 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [term 1 FOLLOWER]: Refusing update from remote peer 8720d1a20e404a5ea8b6d3b50bc2a6e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:31:10.645452 2539 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:31:10.646507 2510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8720d1a20e404a5ea8b6d3b50bc2a6e3. Latest consensus state: current_term: 1 leader_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } } }
I20260925 02:31:10.646587 2510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:31:10.650414 2507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.637014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45606 (local address 127.11.220.252:35587)
0925 02:31:10.637046 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.637048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.637570 (+ 522us) server_negotiation.cc:407] Connection header received
0925 02:31:10.637665 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.637666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.637675 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.637702 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.638476 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.639173 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.640175 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.640470 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.640511 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.647570 (+ 7059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.647572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.647572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.647578 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.649267 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.649629 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.649630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.649631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.649690 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.649877 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.649878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.649878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.650002 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.650070 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.650260 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.650330 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 02:31:10.650672 2523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.636690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.11.220.253:33883)
0925 02:31:10.636727 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.636729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.637522 (+ 793us) server_negotiation.cc:407] Connection header received
0925 02:31:10.637620 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.637620 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.637631 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.637660 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.638333 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.639102 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.642307 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.642727 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.642788 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.647844 (+ 5056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.647846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.647846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.647854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.649496 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.649878 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.649879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.649880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.649930 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.650121 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.650122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.650122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.650235 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.650302 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.650497 (+ 195us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.650588 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:31:10.652886 2539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } } }
I20260925 02:31:10.652961 2539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:10.653002 2539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } } }
I20260925 02:31:10.653076 2539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:10.653498 2525 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } } }
I20260925 02:31:10.653573 2525 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:31:10.653936 2510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8720d1a20e404a5ea8b6d3b50bc2a6e3" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 35641 } } peers { permanent_uuid: "68c12ac78b2b4528b5eaf28f390f7b3d" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 33883 } } peers { permanent_uuid: "c30fe419207e45339059576a350ac3e9" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 35587 } } }
I20260925 02:31:10.654017 2510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:31:10.654117 2566 mvcc.cc:205] Tried to move back new op lower bound from 7333083015751614464 to 7333083015712198656. Current Snapshot: MvccSnapshot[applied={T|T < 7333083015751614464}]
I20260925 02:31:10.655426 2508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:10.637331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34348 (local address 127.11.220.254:35641)
0925 02:31:10.637369 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.637371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.640568 (+ 3197us) server_negotiation.cc:407] Connection header received
0925 02:31:10.640658 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.640659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.640668 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.640696 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.641348 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.642053 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.645230 (+ 3177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:10.645534 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:10.645575 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:10.650722 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:10.650723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:10.650724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:10.650730 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:10.652423 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.654658 (+ 2235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.654659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.654660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.654711 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:10.654886 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:10.654886 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:10.654887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:10.654998 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:10.655093 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:10.655272 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 02:31:10.655344 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 02:31:10.655741 2563 mvcc.cc:205] Tried to move back new op lower bound from 7333083015751614464 to 7333083015712198656. Current Snapshot: MvccSnapshot[applied={T|T < 7333083015751614464}]
I20260925 02:31:10.656694 2559 catalog_manager.cc:1347] Generated new cluster ID: 448ca801e8e3426da5de8c9be0679c7e
I20260925 02:31:10.656752 2559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:31:10.663612 2559 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:31:10.664278 2559 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:31:10.673163 2559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8720d1a20e404a5ea8b6d3b50bc2a6e3: Generated new TSK 0
I20260925 02:31:10.673313 2559 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:31:10.696369 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.694913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.11.220.253:33883)
0925 02:31:10.694951 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.694953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.695767 (+ 814us) server_negotiation.cc:407] Connection header received
0925 02:31:10.695863 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.695864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.695875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.695904 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.696299 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37964: BlockingRecv error: recv got EOF from 127.0.0.1:37964 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 02:31:10.697122 2507 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.695249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45622 (local address 127.11.220.252:35587)
0925 02:31:10.695284 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.695286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.696646 (+ 1360us) server_negotiation.cc:407] Connection header received
0925 02:31:10.696735 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.696736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.696745 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.696773 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.697068 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45622: BlockingRecv error: recv got EOF from 127.0.0.1:45622 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 02:31:10.697829 2508 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.695536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.11.220.254:35641)
0925 02:31:10.695570 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.695572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.697335 (+ 1763us) server_negotiation.cc:407] Connection header received
0925 02:31:10.697424 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.697424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.697433 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.697460 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.697776 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34364: BlockingRecv error: recv got EOF from 127.0.0.1:34364 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 02:31:10.699550 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.698906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.11.220.253:33883)
0925 02:31:10.698936 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.698937 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.699117 (+ 180us) server_negotiation.cc:407] Connection header received
0925 02:31:10.699213 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.699214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.699230 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.699258 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.699513 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37970: BlockingRecv error: recv got EOF from 127.0.0.1:37970 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 02:31:10.702317 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.701623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37974 (local address 127.11.220.253:33883)
0925 02:31:10.701651 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.701653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.701861 (+ 208us) server_negotiation.cc:407] Connection header received
0925 02:31:10.701946 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.701946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.701956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.701994 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.702284 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37974: BlockingRecv error: recv got EOF from 127.0.0.1:37974 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 02:31:10.705832 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.705196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.11.220.253:33883)
0925 02:31:10.705225 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.705226 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.705432 (+ 206us) server_negotiation.cc:407] Connection header received
0925 02:31:10.705510 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.705511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.705519 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.705544 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.705795 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37990: BlockingRecv error: recv got EOF from 127.0.0.1:37990 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 02:31:10.711287 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.710657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37996 (local address 127.11.220.253:33883)
0925 02:31:10.710685 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.710687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.710875 (+ 188us) server_negotiation.cc:407] Connection header received
0925 02:31:10.710953 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.710954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.710962 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.710986 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.711243 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37996: BlockingRecv error: recv got EOF from 127.0.0.1:37996 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260925 02:31:10.721067 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.720279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.11.220.253:33883)
0925 02:31:10.720312 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.720313 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.720622 (+ 309us) server_negotiation.cc:407] Connection header received
0925 02:31:10.720708 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.720709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.720717 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.720743 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.721017 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38004: BlockingRecv error: recv got EOF from 127.0.0.1:38004 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 02:31:10.738713 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.737980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38016 (local address 127.11.220.253:33883)
0925 02:31:10.738010 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.738012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.738247 (+ 235us) server_negotiation.cc:407] Connection header received
0925 02:31:10.738340 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.738341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.738349 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.738377 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.738675 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38016: BlockingRecv error: recv got EOF from 127.0.0.1:38016 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 02:31:10.772531 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.771756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38024 (local address 127.11.220.253:33883)
0925 02:31:10.771793 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.771795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.772015 (+ 220us) server_negotiation.cc:407] Connection header received
0925 02:31:10.772118 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.772119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.772129 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.772155 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.772479 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38024: BlockingRecv error: recv got EOF from 127.0.0.1:38024 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 02:31:10.838426 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.837773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.11.220.253:33883)
0925 02:31:10.837840 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.837842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.837934 (+ 92us) server_negotiation.cc:407] Connection header received
0925 02:31:10.837989 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.837990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.838000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.838038 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.838371 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38040: BlockingRecv error: recv got EOF from 127.0.0.1:38040 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260925 02:31:10.968348 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:10.967576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38042 (local address 127.11.220.253:33883)
0925 02:31:10.967652 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:10.967654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:10.967750 (+ 96us) server_negotiation.cc:407] Connection header received
0925 02:31:10.967849 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:10.967850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:10.967862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:10.967936 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:10.968286 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38042: BlockingRecv error: recv got EOF from 127.0.0.1:38042 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 02:31:11.226063 2523 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:11.225538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38058 (local address 127.11.220.253:33883)
0925 02:31:11.225597 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:11.225598 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:11.225606 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:31:11.225629 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:11.225630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:11.225639 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:11.225683 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:11.225989 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38058: BlockingRecv error: recv got EOF from 127.0.0.1:38058 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 02:31:11.588054 2521 catalog_manager.cc:1259] Loaded cluster ID: 448ca801e8e3426da5de8c9be0679c7e
I20260925 02:31:11.588143 2521 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d: loading cluster ID for follower catalog manager: success
I20260925 02:31:11.589118 2521 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d: acquiring CA information for follower catalog manager: success
I20260925 02:31:11.589435 2521 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68c12ac78b2b4528b5eaf28f390f7b3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:31:11.605427 2536 catalog_manager.cc:1259] Loaded cluster ID: 448ca801e8e3426da5de8c9be0679c7e
I20260925 02:31:11.605489 2536 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9: loading cluster ID for follower catalog manager: success
I20260925 02:31:11.606196 2536 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9: acquiring CA information for follower catalog manager: success
I20260925 02:31:11.606454 2536 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c30fe419207e45339059576a350ac3e9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:31:11.741910 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:11.739268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38068 (local address 127.11.220.253:33883)
0925 02:31:11.739370 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:11.739372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:11.739408 (+ 36us) server_negotiation.cc:407] Connection header received
0925 02:31:11.739507 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:11.739508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:11.739578 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:11.739679 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:31:11.739822 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:11.740369 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:11.741171 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:11.741367 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:11.741427 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:11.741563 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:31:11.741673 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:31:11.741730 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:11.741763 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 02:31:11.741781 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":54,"threads_started":1}
I20260925 02:31:11.745944 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:11.743762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45636 (local address 127.11.220.252:35587)
0925 02:31:11.743867 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:11.743869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:11.743877 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:31:11.743923 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:11.743923 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:11.743956 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:11.744021 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:31:11.744105 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:11.744674 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:11.745330 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:11.745519 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:11.745568 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:11.745695 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:31:11.745783 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:31:11.745836 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:11.745869 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 02:31:11.745886 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260925 02:31:11.750299 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:11.747517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38072 (local address 127.11.220.253:33883)
0925 02:31:11.747603 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:11.747605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:11.747732 (+ 127us) server_negotiation.cc:407] Connection header received
0925 02:31:11.747834 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:11.747834 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:11.747844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:11.747895 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:31:11.748064 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:11.748811 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:11.749602 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:11.749841 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:11.749890 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:11.750046 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:31:11.750076 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:31:11.750123 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:11.750218 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 02:31:11.750241 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 02:31:11.750725 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:11.747507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.11.220.254:35641)
0925 02:31:11.747603 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:11.747606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:11.747745 (+ 139us) server_negotiation.cc:407] Connection header received
0925 02:31:11.747805 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:11.747806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:11.747856 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:11.747938 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:31:11.748107 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:11.748762 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:11.749818 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:11.750128 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:11.750198 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:11.750359 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:31:11.750458 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:31:11.750508 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:11.750610 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 02:31:11.750636 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":55,"threads_started":1}
I20260925 02:31:11.751060 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:11.747661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.11.220.252:35587)
0925 02:31:11.747701 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:11.747703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:11.747920 (+ 217us) server_negotiation.cc:407] Connection header received
0925 02:31:11.748023 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:11.748024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:11.748035 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:11.748084 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:31:11.748255 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:11.748956 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:11.749936 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:11.750206 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:11.750260 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:11.750536 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:31:11.750564 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:31:11.750608 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:11.750923 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 02:31:11.750951 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 02:31:11.757869 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 2279
I20260925 02:31:11.763334 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 2355
I20260925 02:31:11.768251 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 2432
2026-09-25T02:31:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2689 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 02:31:11 dist-test-slave-dn4n krb5kdc[2655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 02:31:11 dist-test-slave-dn4n krb5kdc[2655](info): set up 2 sockets
Sep 25 02:31:11 dist-test-slave-dn4n krb5kdc[2655](info): commencing operation
krb5kdc: starting...
W20260925 02:31:12.753686 12147 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/test-user.keytab.
Sep 25 02:31:12 dist-test-slave-dn4n krb5kdc[2655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T02:31:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T02:31:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.220.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
Entry for principal kudu/127.11.220.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab.
I20260925 02:31:12.855770 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:42719
--webserver_interface=127.11.220.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:42719,127.11.220.253:39993,127.11.220.252:37015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 02:31:12.929167 2668 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:31:12.929332 2668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:12.929346 2668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:12.929363 2668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:12.930347 2668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:12.930382 2668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:12.930393 2668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:12.930424 2668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:12.930447 2668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:12.931746 2668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:42719,127.11.220.253:39993,127.11.220.252:37015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.254.keytab
--principal=kudu/127.11.220.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.254:42719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.220.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:12.931921 2668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:12.932142 2668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 02:31:12.936609 2668 server_base.cc:1034] running on GCE node
W20260925 02:31:12.936578 2675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:12.936568 2674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:12.936578 2677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:12.937016 2668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:12.937331 2668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:12.938490 2668 hybrid_clock.cc:648] HybridClock initialized: now 1790303472938478 us; error 44 us; skew 500 ppm
Sep 25 02:31:12 dist-test-slave-dn4n krb5kdc[2655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303472, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:12.940723 2668 init.cc:377] Logged in from keytab as kudu/127.11.220.254@KRBTEST.COM (short username kudu)
I20260925 02:31:12.941185 2668 webserver.cc:460] Webserver started at http://127.11.220.254:37351/ using document root <none> and password file <none>
I20260925 02:31:12.941347 2668 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:12.941385 2668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:12.941462 2668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:12.942354 2668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/data/instance:
uuid: "975dab5ef5d74a4082fe062667bead10"
format_stamp: "Formatted at 2026-09-25 02:31:12 on dist-test-slave-dn4n"
server_key: "9d1cb70b76d1abb46999feab30d93831"
server_key_iv: "da182327f426e1d946e0b36eabf97a93"
server_key_version: "encryptionkey@0"
I20260925 02:31:12.942629 2668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/wal/instance:
uuid: "975dab5ef5d74a4082fe062667bead10"
format_stamp: "Formatted at 2026-09-25 02:31:12 on dist-test-slave-dn4n"
server_key: "9d1cb70b76d1abb46999feab30d93831"
server_key_iv: "da182327f426e1d946e0b36eabf97a93"
server_key_version: "encryptionkey@0"
I20260925 02:31:12.943813 2668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 02:31:12.944499 2684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:12.944695 2668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 02:31:12.944779 2668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/wal
uuid: "975dab5ef5d74a4082fe062667bead10"
format_stamp: "Formatted at 2026-09-25 02:31:12 on dist-test-slave-dn4n"
server_key: "9d1cb70b76d1abb46999feab30d93831"
server_key_iv: "da182327f426e1d946e0b36eabf97a93"
server_key_version: "encryptionkey@0"
I20260925 02:31:12.944850 2668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:12.955935 2668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:12.957684 2668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:12.957828 2668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:12.961520 2668 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.254:42719
I20260925 02:31:12.961567 2737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.254:42719 every 8 connection(s)
I20260925 02:31:12.961889 2668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/data/info.pb
I20260925 02:31:12.962654 2738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:12.963967 2738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:31:12 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303472, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:31:12.969827 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 2668
I20260925 02:31:12.969924 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-0/wal/instance
I20260925 02:31:12.970103 12147 external_mini_cluster.cc:1428] Setting key b7369d215cfb819e43b3d4811af3121b
I20260925 02:31:12.972702 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:12.964665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.11.220.254:42719)
0925 02:31:12.964820 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:12.964825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:12.964873 (+ 48us) server_negotiation.cc:407] Connection header received
0925 02:31:12.965323 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:12.965325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:12.965341 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:12.965403 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:12.965847 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:12.966658 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:12.967435 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:12.967722 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:12.967792 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:12.969633 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:12.969640 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:12.969640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:12.969645 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:12.971185 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:12.971561 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:12.971564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:12.971565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:12.971652 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:12.971857 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:12.971858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:12.971859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:12.972036 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:12.972113 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:12.972323 (+ 210us) server_negotiation.cc:299] Negotiation successful
0925 02:31:12.972566 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260925 02:31:12.972716 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:12.964617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:42719 (local address 127.0.0.1:33386)
0925 02:31:12.964750 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:12.964789 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:12.965204 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:12.965415 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:12.965416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:12.965644 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:12.965777 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:12.965780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:12.966734 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:12.966735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:12.967369 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:12.967371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:12.967792 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:12.967793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:12.967844 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:12.968221 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:12.968228 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:12.969546 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:12.971285 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:12.971291 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:12.971292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:12.971481 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:12.971715 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:12.971715 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:12.971716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:12.971796 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:12.972106 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:12.972111 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:12.972170 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 02:31:12.972259 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 02:31:12.972434 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260925 02:31:12.973654 2738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:12.973984 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:12.973855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:39993 (local address 127.0.0.1:43520)
0925 02:31:12.973906 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:12.973954 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 02:31:12.974131 2686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.220.253:39993: connect: Connection refused (error 111)
I20260925 02:31:12.974592 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:12.974477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:39993 (local address 127.0.0.1:43530)
0925 02:31:12.974528 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:12.974561 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 02:31:12.974749 2738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.253:39993: Network error: Client connection negotiation failed: client connection to 127.11.220.253:39993: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.220.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab.
Entry for principal kudu/127.11.220.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab.
I20260925 02:31:12.992233 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.253:39993
--webserver_interface=127.11.220.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab
--principal=kudu/127.11.220.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:42719,127.11.220.253:39993,127.11.220.252:37015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:31:13.023998 2738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } attempt: 1
I20260925 02:31:13.024505 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.024327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:39993 (local address 127.0.0.1:43540)
0925 02:31:13.024394 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.024438 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 02:31:13.024896 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.024782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:39993 (local address 127.0.0.1:43556)
0925 02:31:13.024834 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.024866 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 02:31:13.025014 2738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.253:39993: Network error: Client connection negotiation failed: client connection to 127.11.220.253:39993: connect: Connection refused (error 111)
W20260925 02:31:13.067947 2745 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:31:13.068150 2745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:13.068166 2745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:13.068202 2745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:13.069260 2745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:13.069295 2745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:13.069307 2745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:13.069319 2745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:13.069365 2745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:13.070631 2745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:42719,127.11.220.253:39993,127.11.220.252:37015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.253.keytab
--principal=kudu/127.11.220.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.253:39993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.220.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:13.070791 2745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:13.070994 2745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:13.075371 2750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:13.075384 2751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:13.075398 2753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:13.075415 2745 server_base.cc:1034] running on GCE node
I20260925 02:31:13.075771 2745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:13.076004 2745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:13.077131 2745 hybrid_clock.cc:648] HybridClock initialized: now 1790303473077122 us; error 24 us; skew 500 ppm
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:13.079198 2745 init.cc:377] Logged in from keytab as kudu/127.11.220.253@KRBTEST.COM (short username kudu)
I20260925 02:31:13.079504 2745 webserver.cc:460] Webserver started at http://127.11.220.253:44029/ using document root <none> and password file <none>
I20260925 02:31:13.079658 2745 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:13.079694 2745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:13.079771 2745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:13.080772 2745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/data/instance:
uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf"
format_stamp: "Formatted at 2026-09-25 02:31:13 on dist-test-slave-dn4n"
server_key: "a18cf65a03112757f02ee81ce8aabf7c"
server_key_iv: "5264e50a383738abd4b43ca377d1dc03"
server_key_version: "encryptionkey@0"
I20260925 02:31:13.081060 2745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/wal/instance:
uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf"
format_stamp: "Formatted at 2026-09-25 02:31:13 on dist-test-slave-dn4n"
server_key: "a18cf65a03112757f02ee81ce8aabf7c"
server_key_iv: "5264e50a383738abd4b43ca377d1dc03"
server_key_version: "encryptionkey@0"
I20260925 02:31:13.082211 2745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 02:31:13.082809 2760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:13.082984 2745 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:31:13.083052 2745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/wal
uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf"
format_stamp: "Formatted at 2026-09-25 02:31:13 on dist-test-slave-dn4n"
server_key: "a18cf65a03112757f02ee81ce8aabf7c"
server_key_iv: "5264e50a383738abd4b43ca377d1dc03"
server_key_version: "encryptionkey@0"
I20260925 02:31:13.083103 2745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:13.093178 2738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } attempt: 2
I20260925 02:31:13.093535 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.093409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:39993 (local address 127.0.0.1:43558)
0925 02:31:13.093465 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.093504 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 02:31:13.093840 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.093765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:39993 (local address 127.0.0.1:43560)
0925 02:31:13.093802 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.093825 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260925 02:31:13.093951 2738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.253:39993: Network error: Client connection negotiation failed: client connection to 127.11.220.253:39993: connect: Connection refused (error 111)
I20260925 02:31:13.096750 2745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:13.098273 2745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:13.098415 2745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:13.102067 2745 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.253:39993
I20260925 02:31:13.102102 2814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.253:39993 every 8 connection(s)
I20260925 02:31:13.102408 2745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/data/info.pb
I20260925 02:31:13.103156 2815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:13.104400 2815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:13.106065 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 2745
I20260925 02:31:13.106148 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-1/wal/instance
I20260925 02:31:13.106339 12147 external_mini_cluster.cc:1428] Setting key 8ba6dc70293b0d7dda04c236c2809556
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:31:13.113329 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.105107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33396 (local address 127.11.220.254:42719)
0925 02:31:13.105154 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.105155 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.105313 (+ 158us) server_negotiation.cc:407] Connection header received
0925 02:31:13.105763 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.105764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.105775 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.105833 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.106490 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.107223 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.108134 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.108337 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.108418 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.110566 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.110567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.110568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.110574 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.112071 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.112432 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.112434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.112434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.112520 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.112739 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.112741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.112741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.112877 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.112946 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.113163 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.113247 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 02:31:13.113348 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.105056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:42719 (local address 127.0.0.1:33396)
0925 02:31:13.105207 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.105241 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:13.105688 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:13.105879 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:13.105882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:13.106101 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:13.106347 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.106351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.107307 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.107307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.108040 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.108041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.108430 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.108431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.108568 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.109022 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:13.109031 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:13.110483 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:13.112153 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.112160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.112161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.112350 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.112585 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.112586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.112586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.112675 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.112942 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:13.112946 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:13.113040 (+ 94us) client_negotiation.cc:769] Sending connection context
0925 02:31:13.113116 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 02:31:13.113216 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260925 02:31:13.113917 2815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.220.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.220.252@KRBTEST.COM" created.
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
I20260925 02:31:13.120600 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.114176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:39993 (local address 127.0.0.1:43562)
0925 02:31:13.114217 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.114226 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:13.114260 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:13.114561 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:13.114562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:13.114676 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:13.114717 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.114718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.115510 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.115511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.115949 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.115951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.116359 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.116360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.116411 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.116781 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:13.116785 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:13.117762 (+ 977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:13.119567 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.119568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.119568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.119707 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.119916 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.119917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.119917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.120016 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.120341 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:13.120341 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:13.120372 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 02:31:13.120421 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 02:31:13.120539 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 02:31:13.120762 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.114225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.11.220.253:39993)
0925 02:31:13.114385 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.114390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.114415 (+ 25us) server_negotiation.cc:407] Connection header received
0925 02:31:13.114438 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.114441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.114454 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.114560 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.114778 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.115439 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.116013 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.116300 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.116346 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.117845 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.117853 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.117853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.117859 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.119473 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.119783 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.119786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.119786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.119845 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.120088 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.120089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.120090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.120272 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.120357 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.120474 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.120610 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260925 02:31:13.121316 2815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:13.121590 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.121490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43754)
0925 02:31:13.121530 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.121566 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 02:31:13.121737 2764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
I20260925 02:31:13.122115 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.122029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43768)
0925 02:31:13.122080 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.122101 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 02:31:13.122232 2815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:37015: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.220.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab.
Entry for principal kudu/127.11.220.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab.
I20260925 02:31:13.129171 12147 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfV0OY1/build/release/bin/kudu
/tmp/dist-test-taskfV0OY1/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.252:37015
--webserver_interface=127.11.220.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab
--principal=kudu/127.11.220.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.220.212:46713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.220.254:42719,127.11.220.253:39993,127.11.220.252:37015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 02:31:13.171456 2815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } attempt: 1
I20260925 02:31:13.171947 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.171827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43774)
0925 02:31:13.171880 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.171915 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 02:31:13.172329 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.172222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43788)
0925 02:31:13.172268 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.172300 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 02:31:13.172465 2815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:37015: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
I20260925 02:31:13.185119 2738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } attempt: 3
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303472, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
I20260925 02:31:13.191960 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.185421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:39993 (local address 127.0.0.1:43566)
0925 02:31:13.185480 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.185494 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:13.185570 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:13.185717 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:13.185718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:13.185907 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:13.186000 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.186002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.186652 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.186653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.187413 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.187416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.187737 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.187738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.187812 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.188307 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:13.188312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:13.189584 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:13.191004 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.191005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.191006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.191194 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.191378 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.191379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.191379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.191434 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.191664 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:13.191664 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:13.191706 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 02:31:13.191786 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 02:31:13.191874 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 02:31:13.191957 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.185435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.11.220.253:39993)
0925 02:31:13.185475 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.185479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.185571 (+ 92us) server_negotiation.cc:407] Connection header received
0925 02:31:13.185645 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.185645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.185656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.185700 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.186091 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.186584 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.187485 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.187674 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.187718 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.189669 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.189671 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.189671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.189677 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.190927 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.191261 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.191263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.191263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.191316 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.191497 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.191498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.191498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.191600 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.191671 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.191809 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.191874 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 02:31:13.192260 2738 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:13.192576 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.192492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43802)
0925 02:31:13.192520 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.192542 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260925 02:31:13.192863 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.192786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43810)
0925 02:31:13.192825 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.192845 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 02:31:13.192960 2738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:37015: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
W20260925 02:31:13.203343 2822 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 02:31:13.203517 2822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 02:31:13.203532 2822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 02:31:13.203573 2822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 02:31:13.204730 2822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 02:31:13.204767 2822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 02:31:13.204782 2822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 02:31:13.204793 2822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 02:31:13.204804 2822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 02:31:13.206159 2822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.220.212:46713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.220.254:42719,127.11.220.253:39993,127.11.220.252:37015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/krb5kdc/kudu_127.11.220.252.keytab
--principal=kudu/127.11.220.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.220.252:37015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.220.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 01:40:19 UTC on ccycloud.quasar-gxqehx.root.comops.site
build id 4671765
I20260925 02:31:13.206341 2822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 02:31:13.206559 2822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 02:31:13.210769 2828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 02:31:13.210778 2827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:13.210896 2822 server_base.cc:1034] running on GCE node
W20260925 02:31:13.210778 2830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 02:31:13.211141 2822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 02:31:13.211414 2822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 02:31:13.212514 2822 hybrid_clock.cc:648] HybridClock initialized: now 1790303473212497 us; error 32 us; skew 500 ppm
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 02:31:13.214761 2822 init.cc:377] Logged in from keytab as kudu/127.11.220.252@KRBTEST.COM (short username kudu)
I20260925 02:31:13.215179 2822 webserver.cc:460] Webserver started at http://127.11.220.252:38733/ using document root <none> and password file <none>
I20260925 02:31:13.215396 2822 fs_manager.cc:359] Metadata directory not provided
I20260925 02:31:13.215437 2822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 02:31:13.215615 2822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 02:31:13.216670 2822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/data/instance:
uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6"
format_stamp: "Formatted at 2026-09-25 02:31:13 on dist-test-slave-dn4n"
server_key: "17215fcbfb89eb98bc3ac8eb023fe545"
server_key_iv: "fe5c48ac59eee44d626aff86a857251b"
server_key_version: "encryptionkey@0"
I20260925 02:31:13.217002 2822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/wal/instance:
uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6"
format_stamp: "Formatted at 2026-09-25 02:31:13 on dist-test-slave-dn4n"
server_key: "17215fcbfb89eb98bc3ac8eb023fe545"
server_key_iv: "fe5c48ac59eee44d626aff86a857251b"
server_key_version: "encryptionkey@0"
I20260925 02:31:13.218245 2822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 02:31:13.218870 2838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 02:31:13.219054 2822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 02:31:13.219163 2822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/data,/tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/wal
uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6"
format_stamp: "Formatted at 2026-09-25 02:31:13 on dist-test-slave-dn4n"
server_key: "17215fcbfb89eb98bc3ac8eb023fe545"
server_key_iv: "fe5c48ac59eee44d626aff86a857251b"
server_key_version: "encryptionkey@0"
I20260925 02:31:13.219254 2822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 02:31:13.224074 2738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } attempt: 1
I20260925 02:31:13.224388 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.224295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43822)
0925 02:31:13.224339 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.224365 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260925 02:31:13.224726 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.224639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43836)
0925 02:31:13.224680 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.224702 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 02:31:13.224805 2738 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:37015: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
I20260925 02:31:13.240619 2815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } attempt: 2
I20260925 02:31:13.240936 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.240820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43850)
0925 02:31:13.240872 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.240907 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 02:31:13.241251 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.241171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43856)
0925 02:31:13.241211 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.241234 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 02:31:13.241349 2815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.220.252:37015: Network error: Client connection negotiation failed: client connection to 127.11.220.252:37015: connect: Connection refused (error 111)
I20260925 02:31:13.244019 2822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 02:31:13.246157 2822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 02:31:13.246304 2822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 02:31:13.250036 2822 rpc_server.cc:307] RPC server started. Bound to: 127.11.220.252:37015
I20260925 02:31:13.250067 2892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.220.252:37015 every 8 connection(s)
I20260925 02:31:13.250380 2822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/data/info.pb
I20260925 02:31:13.251173 2893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 02:31:13.252610 2893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:13.253144 12147 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfV0OY1/build/release/bin/kudu as pid 2822
I20260925 02:31:13.253252 12147 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfV0OY1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790303222243611-12147-0/minicluster-data/master-2/wal/instance
I20260925 02:31:13.253465 12147 external_mini_cluster.cc:1428] Setting key 3d0b75e1d1a3c1b29610e2c12815cf6f
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
I20260925 02:31:13.261394 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.253257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.11.220.254:42719)
0925 02:31:13.253299 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.253301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.253518 (+ 217us) server_negotiation.cc:407] Connection header received
0925 02:31:13.254107 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.254109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.254120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.254165 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.254663 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.255116 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.256051 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.256239 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.256317 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.258489 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.258490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.258490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.258496 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.259971 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.260431 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.260432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.260432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.260556 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.260811 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.260812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.260812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.260902 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.260976 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.261246 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.261317 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 02:31:13.261696 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.253204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.254:42719 (local address 127.0.0.1:33402)
0925 02:31:13.253379 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.253411 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:13.254001 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:13.254181 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:13.254185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:13.254404 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:13.254565 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.254569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.255194 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.255195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.255937 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.255940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.256327 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.256328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.256486 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.256936 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:13.256946 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:13.258400 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:13.260060 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.260068 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.260070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.260357 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.260622 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.260623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.260623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.260746 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.260989 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:13.260996 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:13.261115 (+ 119us) client_negotiation.cc:769] Sending connection context
0925 02:31:13.261226 (+ 111us) client_negotiation.cc:240] Negotiation successful
0925 02:31:13.261350 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":75,"thread_start_us":68,"threads_started":1}
I20260925 02:31:13.262095 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.253868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.11.220.252:37015)
0925 02:31:13.254001 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.254006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.254055 (+ 49us) server_negotiation.cc:407] Connection header received
0925 02:31:13.254130 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.254131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.254145 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.254223 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.254826 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.255591 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.256252 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.256595 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.256675 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.259014 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.259020 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.259021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.259027 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.260870 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.261213 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.261215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.261216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.261266 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.261451 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.261452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.261452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.261579 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.261711 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.261922 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.261997 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260925 02:31:13.262205 2893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.254@KRBTEST.COM
I20260925 02:31:13.268312 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.262424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.253:39993 (local address 127.0.0.1:43578)
0925 02:31:13.262459 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.262467 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:13.262498 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:13.262728 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:13.262729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:13.262845 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:13.262902 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.262903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.263491 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.263493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.263938 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.263940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.264304 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.264305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.264368 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.264763 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:13.264767 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:13.265768 (+ 1001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:13.267386 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.267387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.267387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.267570 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.267767 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.267768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.267769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.267840 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.268112 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:13.268113 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:13.268145 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 02:31:13.268198 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 02:31:13.268258 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260925 02:31:13.268378 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.262543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43578 (local address 127.11.220.253:39993)
0925 02:31:13.262584 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.262586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.262593 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:31:13.262614 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.262615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.262625 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.262670 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.262983 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.263415 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.264019 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.264234 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.264306 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.265848 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.265849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.265850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.265856 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.267270 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.267638 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.267639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.267640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.267697 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.267905 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.267906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.267906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.268040 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.268116 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.268222 (+ 106us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.268290 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 02:31:13.268707 2893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } has no permanent_uuid. Determining permanent_uuid...
I20260925 02:31:13.269752 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.262728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.11.220.254:42719)
0925 02:31:13.262773 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.262775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.262823 (+ 48us) server_negotiation.cc:407] Connection header received
0925 02:31:13.262906 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.262907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.262915 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.262955 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.263381 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.263788 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.264423 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.264743 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.264814 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.266974 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.266975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.266976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.266994 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.268691 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.269082 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.269083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.269083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.269139 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.269323 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.269324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.269324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.269445 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.269572 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.269645 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.269699 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.252@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.220.253@KRBTEST.COM
I20260925 02:31:13.275880 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.268993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.11.220.252:37015)
0925 02:31:13.269036 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.269038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.269062 (+ 24us) server_negotiation.cc:407] Connection header received
0925 02:31:13.269082 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.269082 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.269093 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.269135 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.269396 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.270127 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.270889 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.271193 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.271262 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.273052 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.273052 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.273053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.273058 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.274430 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.274806 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.274807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.274807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.274862 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.275096 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.275096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.275097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.275269 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.275355 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.275635 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.275727 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 02:31:13.276077 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.268932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43870)
0925 02:31:13.268976 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.268986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:13.269032 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:13.269139 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:13.269140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:13.269277 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:13.269327 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.269328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.270200 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.270201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.270793 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.270795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.271259 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.271260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.271305 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.271709 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:13.271713 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:13.272981 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:13.274524 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.274525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.274526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.274737 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.274949 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.274950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.274951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.275012 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.275379 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:13.275380 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:13.275443 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 02:31:13.275559 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 02:31:13.276006 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260925 02:31:13.277601 2893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6: Bootstrap starting.
I20260925 02:31:13.278370 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.270593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.11.220.253:39993)
0925 02:31:13.270621 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.270623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.270956 (+ 333us) server_negotiation.cc:407] Connection header received
0925 02:31:13.271012 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.271013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.271021 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.271060 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.271650 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.272148 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.273111 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.273408 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.273470 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.275805 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.275807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.275807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.275825 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.277391 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.277706 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.277707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.277707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.277758 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.277908 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.277909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.277909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.278035 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.278203 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.278239 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.278300 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260925 02:31:13.278503 2893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:13.278798 2893 log.cc:824] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:13.279515 2893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6: No bootstrap required, opened a new log
I20260925 02:31:13.279640 12147 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 02:31:13.280972 2893 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } }
I20260925 02:31:13.280427 2712 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "975dab5ef5d74a4082fe062667bead10"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33410
I20260925 02:31:13.281096 2893 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8fd305ddfcd4c1aa2d552865c1bcad6, State: Initialized, Role: FOLLOWER
I20260925 02:31:13.281175 2893 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } }
I20260925 02:31:13.281446 2899 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } } }
I20260925 02:31:13.281502 2893 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:13.281517 2899 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:31:13.283090 2910 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:31:13.283141 2910 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:31:13.283355 2822 master_runner.cc:186] Error getting master registration for 127.11.220.254:42719: OK, instance_id { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" instance_seqno: 1790303472960548 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:31:13.286054 2822 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:31:13.299932 2738 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } attempt: 2
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303472, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.254@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
I20260925 02:31:13.306795 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.300158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43880)
0925 02:31:13.300197 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.300206 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:13.300237 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:13.300485 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:13.300486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:13.300646 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:13.300684 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.300685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.301500 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.301501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.302199 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.302200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.302642 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.302643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.302693 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.303050 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:13.303055 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:13.304119 (+ 1064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:13.305784 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.305785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.305786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.305995 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.306186 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.306187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.306187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.306245 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.306515 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:13.306516 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:13.306557 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 02:31:13.306620 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 02:31:13.306701 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 02:31:13.306806 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.300309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43880 (local address 127.11.220.252:37015)
0925 02:31:13.300354 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.300356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.300363 (+ 7us) server_negotiation.cc:407] Connection header received
0925 02:31:13.300380 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.300381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.300388 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.300748 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.300774 (+ 26us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.301417 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.302277 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.302570 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.302634 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.304215 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.304217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.304217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.304223 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.305706 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.306064 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.306065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.306066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.306120 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.306306 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.306306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.306307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.306451 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.306516 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.306657 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.306727 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 02:31:13.308192 2738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10: Bootstrap starting.
I20260925 02:31:13.308841 2738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:13.309103 2738 log.cc:824] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:13.309846 2738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10: No bootstrap required, opened a new log
I20260925 02:31:13.311031 2738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } }
I20260925 02:31:13.311172 2738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 975dab5ef5d74a4082fe062667bead10, State: Initialized, Role: FOLLOWER
I20260925 02:31:13.311247 2738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } }
I20260925 02:31:13.311522 2738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:13.311589 2712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 02:31:13.311518 2914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } } }
I20260925 02:31:13.311657 2712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:13.311707 2914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:31:13.312496 2712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } }
I20260925 02:31:13.312724 2712 leader_election.cc:288] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [CANDIDATE]: Term 1 election: Requested vote from peers fa9ff2ec70b64eb8bb2104b7ce72ddcf (127.11.220.253:39993), f8fd305ddfcd4c1aa2d552865c1bcad6 (127.11.220.252:37015)
I20260925 02:31:13.312958 2867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "975dab5ef5d74a4082fe062667bead10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6"
I20260925 02:31:13.313050 2867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:13.313781 2867 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 975dab5ef5d74a4082fe062667bead10 in term 1.
I20260925 02:31:13.313992 2688 leader_election.cc:302] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 975dab5ef5d74a4082fe062667bead10, f8fd305ddfcd4c1aa2d552865c1bcad6; no voters:
I20260925 02:31:13.314107 2914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 02:31:13.314564 2914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [term 1 LEADER]: Becoming Leader. State: Replica: 975dab5ef5d74a4082fe062667bead10, State: Running, Role: LEADER
I20260925 02:31:13.314668 2914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } }
W20260925 02:31:13.315369 2925 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:31:13.315418 2925 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:31:13.316032 2668 master_runner.cc:186] Error getting master registration for 127.11.220.253:39993: OK, instance_id { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" instance_seqno: 1790303473101031 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 02:31:13.318120 2668 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 02:31:13.318324 2931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 975dab5ef5d74a4082fe062667bead10. Latest consensus state: current_term: 1 leader_uuid: "975dab5ef5d74a4082fe062667bead10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } } }
I20260925 02:31:13.318421 2931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:13.318678 2936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 02:31:13.319007 2936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 02:31:13.322584 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.313829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33418 (local address 127.11.220.254:42719)
0925 02:31:13.313882 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.313884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.314958 (+ 1074us) server_negotiation.cc:407] Connection header received
0925 02:31:13.315065 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.315065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.315075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.315103 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.315825 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.316497 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.317434 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.317753 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.317825 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.319364 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.319365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.319365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.319372 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.320856 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.321453 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.321454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.321454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.321520 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.321836 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.321837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.321837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.321965 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.322030 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.322423 (+ 393us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.322497 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 02:31:13.326169 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.314474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43894 (local address 127.11.220.252:37015)
0925 02:31:13.314504 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.314506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.318642 (+ 4136us) server_negotiation.cc:407] Connection header received
0925 02:31:13.318741 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.318742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.318752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.318784 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.319507 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.320160 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.321174 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.321494 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.321571 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.323213 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.323214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.323214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.323221 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.324743 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.325125 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.325126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.325127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.325196 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.325521 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.325522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.325522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.325657 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.325724 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.326037 (+ 313us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.326105 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 02:31:13.328972 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.314835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43592 (local address 127.11.220.253:39993)
0925 02:31:13.314918 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.314920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.319315 (+ 4395us) server_negotiation.cc:407] Connection header received
0925 02:31:13.321282 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.321283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.321292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.321318 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.321908 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.322645 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.324001 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.324645 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.324717 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.326105 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.326106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.326107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.326113 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.327897 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.328249 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.328250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.328250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.328304 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.328520 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.328521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.328522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.328646 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.328743 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.328826 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.328892 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 02:31:13.332499 2815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } attempt: 3
Sep 25 02:31:13 dist-test-slave-dn4n krb5kdc[2655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790303473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.220.253@KRBTEST.COM for kudu/127.11.220.252@KRBTEST.COM
I20260925 02:31:13.339510 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.332690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.220.252:37015 (local address 127.0.0.1:43908)
0925 02:31:13.332729 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 02:31:13.332737 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 02:31:13.332766 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 02:31:13.333005 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 02:31:13.333006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 02:31:13.333143 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0925 02:31:13.333176 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.333177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.334072 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.334073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.334788 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 02:31:13.334790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.335200 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.335201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 02:31:13.335251 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.335598 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 02:31:13.335602 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 02:31:13.336869 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 02:31:13.338483 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.338484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.338484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.338692 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.338868 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 02:31:13.338869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 02:31:13.338869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 02:31:13.338915 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 02:31:13.339238 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 02:31:13.339239 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 02:31:13.339279 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 02:31:13.339344 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 02:31:13.339426 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 02:31:13.339601 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:13.332824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43908 (local address 127.11.220.252:37015)
0925 02:31:13.332872 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.332874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.332880 (+ 6us) server_negotiation.cc:407] Connection header received
0925 02:31:13.332897 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.332912 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.332920 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.332970 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.333244 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.334003 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.334866 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:13.335127 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:13.335194 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:13.336939 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 02:31:13.336940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 02:31:13.336941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 02:31:13.336947 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 02:31:13.338385 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.338766 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.338766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.338767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.338811 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 02:31:13.338975 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 02:31:13.338976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 02:31:13.338977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 02:31:13.339160 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 02:31:13.339247 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:13.339375 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 02:31:13.339482 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 02:31:13.340849 2815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf: Bootstrap starting.
I20260925 02:31:13.341470 2815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf: Neither blocks nor log segments found. Creating new log.
I20260925 02:31:13.341737 2815 log.cc:824] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf: Log is configured to *not* fsync() on all Append() calls
I20260925 02:31:13.342455 2815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf: No bootstrap required, opened a new log
I20260925 02:31:13.344342 2815 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } }
I20260925 02:31:13.344497 2815 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9ff2ec70b64eb8bb2104b7ce72ddcf, State: Initialized, Role: FOLLOWER
I20260925 02:31:13.344584 2815 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } }
I20260925 02:31:13.344832 2939 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } } }
I20260925 02:31:13.344961 2939 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:31:13.345582 2867 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [term 1 FOLLOWER]: Refusing update from remote peer 975dab5ef5d74a4082fe062667bead10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:31:13.345697 2788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "975dab5ef5d74a4082fe062667bead10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf"
I20260925 02:31:13.345642 2815 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [sys.catalog]: configured and running, proceeding with master startup.
I20260925 02:31:13.345810 2787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [term 0 FOLLOWER]: Advancing to term 1
I20260925 02:31:13.346590 2787 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [term 1 FOLLOWER]: Refusing update from remote peer 975dab5ef5d74a4082fe062667bead10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 02:31:13.346899 2931 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:31:13.347090 2914 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 02:31:13.347381 2788 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 975dab5ef5d74a4082fe062667bead10 in term 1.
I20260925 02:31:13.348318 2899 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 975dab5ef5d74a4082fe062667bead10. Latest consensus state: current_term: 1 leader_uuid: "975dab5ef5d74a4082fe062667bead10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } } }
I20260925 02:31:13.348412 2899 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:31:13.348352 2939 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 975dab5ef5d74a4082fe062667bead10. Latest consensus state: current_term: 1 leader_uuid: "975dab5ef5d74a4082fe062667bead10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } } }
I20260925 02:31:13.348574 2939 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [sys.catalog]: This master's current role is: FOLLOWER
W20260925 02:31:13.348922 2766 tablet.cc:2369] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 02:31:13.353034 2954 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 02:31:13.353137 2954 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 02:31:13.353938 2914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "975dab5ef5d74a4082fe062667bead10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } } }
I20260925 02:31:13.354008 2914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:13.354071 2937 mvcc.cc:205] Tried to move back new op lower bound from 7333083026820096000 to 7333083026697510912. Current Snapshot: MvccSnapshot[applied={T|T < 7333083026820096000}]
I20260925 02:31:13.354656 2939 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "975dab5ef5d74a4082fe062667bead10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } } }
I20260925 02:31:13.354727 2939 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:31:13.354847 2945 mvcc.cc:205] Tried to move back new op lower bound from 7333083026820096000 to 7333083026697510912. Current Snapshot: MvccSnapshot[applied={T|T < 7333083026820096000}]
I20260925 02:31:13.355822 2931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "975dab5ef5d74a4082fe062667bead10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } } }
I20260925 02:31:13.355909 2931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10 [sys.catalog]: This master's current role is: LEADER
I20260925 02:31:13.356323 2899 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "975dab5ef5d74a4082fe062667bead10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "975dab5ef5d74a4082fe062667bead10" member_type: VOTER last_known_addr { host: "127.11.220.254" port: 42719 } } peers { permanent_uuid: "fa9ff2ec70b64eb8bb2104b7ce72ddcf" member_type: VOTER last_known_addr { host: "127.11.220.253" port: 39993 } } peers { permanent_uuid: "f8fd305ddfcd4c1aa2d552865c1bcad6" member_type: VOTER last_known_addr { host: "127.11.220.252" port: 37015 } } }
I20260925 02:31:13.356387 2899 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 02:31:13.356650 2947 mvcc.cc:205] Tried to move back new op lower bound from 7333083026820096000 to 7333083026697510912. Current Snapshot: MvccSnapshot[applied={T|T < 7333083026820096000}]
I20260925 02:31:13.357486 2936 catalog_manager.cc:1347] Generated new cluster ID: 269f7bc15efa464897ed5e66e24ff4aa
I20260925 02:31:13.357527 2936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 02:31:13.366937 2936 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 02:31:13.367622 2936 catalog_manager.cc:1504] Loading token signing keys...
I20260925 02:31:13.373276 2936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 975dab5ef5d74a4082fe062667bead10: Generated new TSK 0
I20260925 02:31:13.373474 2936 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 02:31:13.404651 2741 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.403398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33424 (local address 127.11.220.254:42719)
0925 02:31:13.403432 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.403434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.403982 (+ 548us) server_negotiation.cc:407] Connection header received
0925 02:31:13.404094 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.404096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.404106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.404135 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.404586 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33424: BlockingRecv error: recv got EOF from 127.0.0.1:33424 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 02:31:13.405385 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.403199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.11.220.253:39993)
0925 02:31:13.403239 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.403241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.404890 (+ 1649us) server_negotiation.cc:407] Connection header received
0925 02:31:13.404984 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.404985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.404995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.405023 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.405327 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43598: BlockingRecv error: recv got EOF from 127.0.0.1:43598 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 02:31:13.406061 2897 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.403588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.11.220.252:37015)
0925 02:31:13.403630 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.403632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.405596 (+ 1964us) server_negotiation.cc:407] Connection header received
0925 02:31:13.405693 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.405694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.405703 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.405728 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.405994 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43924: BlockingRecv error: recv got EOF from 127.0.0.1:43924 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 02:31:13.408442 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.407539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.11.220.253:39993)
0925 02:31:13.407586 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.407588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.407957 (+ 369us) server_negotiation.cc:407] Connection header received
0925 02:31:13.408064 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.408065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.408083 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.408111 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.408399 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43606: BlockingRecv error: recv got EOF from 127.0.0.1:43606 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260925 02:31:13.411280 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.410576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43610 (local address 127.11.220.253:39993)
0925 02:31:13.410606 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.410607 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.410838 (+ 231us) server_negotiation.cc:407] Connection header received
0925 02:31:13.410922 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.410923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.410932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.410957 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.411242 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43610: BlockingRecv error: recv got EOF from 127.0.0.1:43610 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 02:31:13.417476 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.416737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43622 (local address 127.11.220.253:39993)
0925 02:31:13.416769 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.416771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.417016 (+ 245us) server_negotiation.cc:407] Connection header received
0925 02:31:13.417118 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.417119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.417129 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.417156 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.417435 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43622: BlockingRecv error: recv got EOF from 127.0.0.1:43622 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 02:31:13.423245 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.422526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43634 (local address 127.11.220.253:39993)
0925 02:31:13.422554 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.422556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.422748 (+ 192us) server_negotiation.cc:407] Connection header received
0925 02:31:13.422831 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.422832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.422841 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.422865 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.423187 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43634: BlockingRecv error: recv got EOF from 127.0.0.1:43634 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260925 02:31:13.432984 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.432271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.11.220.253:39993)
0925 02:31:13.432303 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.432304 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.432559 (+ 255us) server_negotiation.cc:407] Connection header received
0925 02:31:13.432645 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.432646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.432655 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.432681 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.432949 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43648: BlockingRecv error: recv got EOF from 127.0.0.1:43648 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 02:31:13.450641 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.450083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43656 (local address 127.11.220.253:39993)
0925 02:31:13.450117 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.450119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.450229 (+ 110us) server_negotiation.cc:407] Connection header received
0925 02:31:13.450286 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.450287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.450297 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.450343 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.450596 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43656: BlockingRecv error: recv got EOF from 127.0.0.1:43656 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 02:31:13.484267 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.483592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43666 (local address 127.11.220.253:39993)
0925 02:31:13.483631 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.483633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.483752 (+ 119us) server_negotiation.cc:407] Connection header received
0925 02:31:13.483818 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.483819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.483829 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.483906 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.484206 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43666: BlockingRecv error: recv got EOF from 127.0.0.1:43666 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 02:31:13.549896 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.549300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43680 (local address 127.11.220.253:39993)
0925 02:31:13.549365 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.549369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.549441 (+ 72us) server_negotiation.cc:407] Connection header received
0925 02:31:13.549472 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.549473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.549483 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.549542 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.549841 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43680: BlockingRecv error: recv got EOF from 127.0.0.1:43680 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260925 02:31:13.679805 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.679072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.11.220.253:39993)
0925 02:31:13.679151 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.679153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.679247 (+ 94us) server_negotiation.cc:407] Connection header received
0925 02:31:13.679339 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.679341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.679352 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.679420 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.679742 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43694: BlockingRecv error: recv got EOF from 127.0.0.1:43694 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 02:31:13.937871 2819 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 02:31:13.937205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43710 (local address 127.11.220.253:39993)
0925 02:31:13.937279 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:13.937281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:13.937291 (+ 10us) server_negotiation.cc:407] Connection header received
0925 02:31:13.937374 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:13.937375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:13.937393 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:13.937456 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 02:31:13.937806 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43710: BlockingRecv error: recv got EOF from 127.0.0.1:43710 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 02:31:14.283670 2910 catalog_manager.cc:1259] Loaded cluster ID: 269f7bc15efa464897ed5e66e24ff4aa
I20260925 02:31:14.283726 2910 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6: loading cluster ID for follower catalog manager: success
I20260925 02:31:14.284451 2910 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6: acquiring CA information for follower catalog manager: success
I20260925 02:31:14.284740 2910 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f8fd305ddfcd4c1aa2d552865c1bcad6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:31:14.353660 2954 catalog_manager.cc:1259] Loaded cluster ID: 269f7bc15efa464897ed5e66e24ff4aa
I20260925 02:31:14.353717 2954 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf: loading cluster ID for follower catalog manager: success
I20260925 02:31:14.354496 2954 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf: acquiring CA information for follower catalog manager: success
I20260925 02:31:14.354723 2954 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fa9ff2ec70b64eb8bb2104b7ce72ddcf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 02:31:14.453665 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:14.451094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.11.220.253:39993)
0925 02:31:14.451205 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:14.451206 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:14.451217 (+ 11us) server_negotiation.cc:407] Connection header received
0925 02:31:14.451293 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:14.451294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:14.451348 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:14.451416 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:31:14.451555 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:14.452045 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:14.452889 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:14.453090 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:14.453144 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:14.453296 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:31:14.453403 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:31:14.453458 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:14.453518 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 02:31:14.453535 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
I20260925 02:31:14.457696 3035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:14.455291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.11.220.252:37015)
0925 02:31:14.455402 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:14.455404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:14.455412 (+ 8us) server_negotiation.cc:407] Connection header received
0925 02:31:14.455484 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:14.455485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:14.455532 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:14.455618 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:31:14.455726 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:14.456168 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:14.456940 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:14.457164 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:14.457232 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:14.457379 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:31:14.457473 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:31:14.457548 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:14.457598 (+ 50us) server_negotiation.cc:299] Negotiation successful
0925 02:31:14.457616 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":49,"threads_started":1}
I20260925 02:31:14.461653 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:14.459161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43732 (local address 127.11.220.253:39993)
0925 02:31:14.459203 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:14.459205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:14.459265 (+ 60us) server_negotiation.cc:407] Connection header received
0925 02:31:14.459327 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:14.459328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:14.459340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:14.459386 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:31:14.459524 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:14.460061 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:14.460889 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:14.461181 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:14.461243 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:14.461372 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:31:14.461405 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:31:14.461464 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:14.461551 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 02:31:14.461572 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 02:31:14.462059 3035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:14.459136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43944 (local address 127.11.220.252:37015)
0925 02:31:14.459168 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:14.459170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:14.459372 (+ 202us) server_negotiation.cc:407] Connection header received
0925 02:31:14.459587 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:14.459588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:14.459599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:14.459648 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:31:14.459761 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:14.460459 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:14.461317 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:14.461590 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:14.461651 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:14.461781 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:31:14.461809 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:31:14.461852 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:14.461974 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 02:31:14.461997 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 02:31:14.462407 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 02:31:14.459079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33430 (local address 127.11.220.254:42719)
0925 02:31:14.459225 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 02:31:14.459227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 02:31:14.459348 (+ 121us) server_negotiation.cc:407] Connection header received
0925 02:31:14.459457 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 02:31:14.459458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 02:31:14.459508 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 02:31:14.459545 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 02:31:14.459724 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:14.460518 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:14.461455 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 02:31:14.461740 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 02:31:14.461791 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 02:31:14.462025 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 02:31:14.462136 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 02:31:14.462200 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 02:31:14.462291 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 02:31:14.462317 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":46,"threads_started":1}
I20260925 02:31:14.463826 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 2668
I20260925 02:31:14.469522 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 2745
I20260925 02:31:14.474435 12147 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfV0OY1/build/release/bin/kudu with pid 2822
2026-09-25T02:31:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2706 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5395 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (252238 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST